-- Logs begin at Tue 2022-05-10 13:12:32 UTC, end at Tue 2022-05-10 15:06:59 UTC. -- May 10 13:55:46.620763 dv-cinder-841125-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 10 13:55:46.630364 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 10 13:55:46.633023 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 10 13:55:46 2022] *** May 10 13:55:46.633167 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 10 13:55:46.633283 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 10 13:55:46.633391 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: nodename: dv-cinder-841125-4-iscsi May 10 13:55:46.633481 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: machine: x86_64 May 10 13:55:46.633592 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: clock source: unix May 10 13:55:46.633696 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: pcre jit disabled May 10 13:55:46.633799 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: detected number of CPU cores: 4 May 10 13:55:46.633904 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: current working directory: / May 10 13:55:46.634007 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: detected binary path: /usr/bin/uwsgi-core May 10 13:55:46.634117 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: your processes number limit is 47680 May 10 13:55:46.634241 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: your memory page size is 4096 bytes May 10 13:55:46.634357 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: detected max file descriptor number: 2048 May 10 13:55:46.634472 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: lock engine: pthread robust mutexes May 10 13:55:46.634733 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: thunder lock: enabled May 10 13:55:46.635025 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 10 13:55:46.635228 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 10 13:55:46.680172 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: Python main interpreter initialized at 0x55d64eea29c0 May 10 13:55:46.680172 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: python threads support enabled May 10 13:55:46.680172 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: your server socket listen backlog is limited to 100 connections May 10 13:55:46.680172 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: your mercy for graceful operations on workers is 90 seconds May 10 13:55:46.680571 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: mapped 403077 bytes (393 KB) for 2 cores May 10 13:55:46.680647 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: *** Operational MODE: preforking *** May 10 13:55:46.680907 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: *** uWSGI is running in multiple interpreter mode *** May 10 13:55:46.680907 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: spawned uWSGI master process (pid: 80404) May 10 13:55:46.681125 dv-cinder-841125-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 10 13:55:46.684335 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: spawned uWSGI worker 1 (pid: 80413, cores: 1) May 10 13:55:46.684335 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: spawned uWSGI worker 2 (pid: 80414, cores: 1) May 10 13:55:46.684335 dv-cinder-841125-4-iscsi devstack@keystone.service[80404]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 10 13:55:52.546829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 13:55:52.546829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: from cryptography.utils import int_from_bytes May 10 13:55:52.547641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 13:55:52.547641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: from cryptography.utils import int_from_bytes May 10 13:55:52.548168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 13:55:52.548168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: from cryptography.utils import int_from_bytes May 10 13:55:52.548648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 10 13:55:52.548648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: from cryptography.utils import int_from_bytes May 10 13:55:52.597710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 13:55:52.597942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 10 13:55:52.598026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 13:55:52.598123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 10 13:55:52.598225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] command line args: [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 13:55:52.598312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] command line args: [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 10 13:55:52.598391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 13:55:52.598471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 10 13:55:52.598553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ================================================================================ {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 13:55:52.598661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ================================================================================ {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 10 13:55:52.598752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] admin_token = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.598919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] admin_token = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] config_dir = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] config_dir = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] config_source = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] debug = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] config_source = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] debug = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.599988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.600993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_config_append = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_config_append = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_dir = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_dir = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_file = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_file = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.601964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.602943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.603903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.604891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] publish_errors = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] publish_errors = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.605968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.606980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.607097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.607210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] standard_threads = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.607926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] standard_threads = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.607926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.608238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.608440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.608631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.608822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.608945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] use_journal = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] use_journal = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] use_json = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] use_json = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] use_stderr = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] use_syslog = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] use_stderr = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] use_syslog = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.609967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.610088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 10 13:55:52.610219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.610929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.611972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.612970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.613918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.614998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.615950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.616886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.617988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.618975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.619911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.620928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.621972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.622823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.623563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.623744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.623944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.624912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.625920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.626946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.627902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.628945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.629895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.630983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.631890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.632924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.633958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634602 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.634960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.635996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.external = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.external = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.636927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.password = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] auth.token = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.password = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] auth.token = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.637975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.638930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.639920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.640937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.641958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.642950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.643995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.644940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.645997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.646936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.647936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.648933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.649985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.650948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.651981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.652920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.653961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654836 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.654977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.655917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.656951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.657885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.658985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.659557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.659686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.659810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.659927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.660971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.661903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.662961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.663965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.664911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.665937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.666942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.667980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668207 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.668900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.669663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.670624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.670822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] role.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] role.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.671954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] role.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] role.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.672893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.673937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.674993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.675893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.676941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.677983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.678938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.679980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.680907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.681957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.682955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.connection = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.683892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.connection = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.684958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.685057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.685154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.685249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.685444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.686940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.687888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80413) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 13:55:52.688896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 10 13:55:52.688896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 10 13:55:52.718382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.718635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.718833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.719359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.719484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.719615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.727913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.728641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.758415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.758651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.758789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.758964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.814521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.814950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.867280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.867516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.868551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.868788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.936333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.936577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.936759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.936847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.936979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.937882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.961716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.961898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.962846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.963019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.963110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.963237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.974807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.975191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.989735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.989963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.990982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.991116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80413) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.991198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:52.991361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 10 13:55:53.009110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.009318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.009412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 13:55:53.009534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 13:55:53.009662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.009865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 13:55:53.009967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 10 13:55:53.010157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.010985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.011159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.022453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.022729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.022982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.023071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.023260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.023342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.023529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.023814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.031939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.032819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.035230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.035514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.035797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.036232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.036988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.037942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.038070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.038240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.038520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.038809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.039081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.066051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.067111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.067635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.067916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.068763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.069049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.073313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.073596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.073880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.074524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.074804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.075085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.081930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.082221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.083035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.083322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.086878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.087169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.087452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.087737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.087988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.088070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.088902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.089090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.089368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.089642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.089912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.108899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.109172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.109765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.110049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.113687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.114623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.117304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.117599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.118426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.118726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.122828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.124290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.126432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.127890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.130126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.130451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.130739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.131023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.131301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.131581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.131742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.132055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.132345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.132635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.132918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.133201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.150566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.150965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.151233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.151502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.160089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.160935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.161342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.161620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.161814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.161898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.162093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.165171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.165452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.165734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.166012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.166321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.169853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.172141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.172419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.175501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.175786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.176079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.176366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.176646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.180329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.180614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.180895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.181178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.181459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.181747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.182020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.190171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.190485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.190773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.191066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.191346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.191639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.191916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.200198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.204276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.207591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.209192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.210587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.210853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.211131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.211420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.213886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.218082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.219446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.220699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.221055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.222741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.223110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.223407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.224309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.224597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.224878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.225155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.231752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.233426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.234693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.234995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.235282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.236733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.237025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.237311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.237588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.243346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80413) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.243629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.243911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.244188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.244462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.244749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.245022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.245295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.245562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.245838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.246108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.246665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.246939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.247137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.247229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80413) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.247447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.247724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.255972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 10 13:55:53.256253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.256540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.256821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.257096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.257387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.257662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.257946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.258236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.258521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.258796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.259077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.259355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.259615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 10 13:55:53.287308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d64eea29c0 pid: 80413 (default app) May 10 13:55:53.290783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966eb5-8ef8-4fec-be3d-4e209054a51c None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:55:53.291094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966eb5-8ef8-4fec-be3d-4e209054a51c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:55:53.291378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966eb5-8ef8-4fec-be3d-4e209054a51c None None] PATH_INFO: `/v3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:55:53.295830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Tue May 10 13:55:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:55:53.300206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d64eea29c0 pid: 80414 (default app) May 10 13:56:02.601745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170677-1677-4e66-a637-3abd14102c16 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:02.602346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170677-1677-4e66-a637-3abd14102c16 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:02.602346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45170677-1677-4e66-a637-3abd14102c16 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:02.603692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 2/2] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:02.611585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:02.612065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:02.612156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:02.643820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG oslo_db.sqlalchemy.engines [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80414) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 13:56:02.831182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG passlib.handlers.bcrypt [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80414) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 13:56:02.831612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG passlib.handlers.bcrypt [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80414) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 13:56:02.888160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:02.890517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:03.017909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1/3] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 10 13:56:03.194251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170677-1677-4e66-a637-3abd14102c16 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:03.228518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG oslo_db.sqlalchemy.engines [None req-45170677-1677-4e66-a637-3abd14102c16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80413) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 10 13:56:03.360066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45170677-1677-4e66-a637-3abd14102c16 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:03.364565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:03.365225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:03.365506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:03.365780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:03.374921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:03.429682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 13:56:03.430024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2719d-24ea-487d-90ba-a369edc9310a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:03.440838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 3/4] 10.222.0.58 () {58 vars in 1324 bytes} [Tue May 10 13:56:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 13:56:03.446922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:03.464611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127e1cd-15c0-4f77-a7c8-fcbc4947725e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:03.469380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:03.470571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:03.470862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:03.471143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] PATH_INFO: `/v3/projects/fab88b77b7fd44528a3ca6841d9f273b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:03.472759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] RBAC: Authorizing `identity:get_project(project_id=fab88b77b7fd44528a3ca6841d9f273b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:03.524214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 10 13:56:03.524556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759c81a-b7ec-49fb-94c8-5e49023401d9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:03.529618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 2/5] 10.222.0.58 () {56 vars in 1207 bytes} [Tue May 10 13:56:03 2022] GET /identity/v3/projects/fab88b77b7fd44528a3ca6841d9f273b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:05.752225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ffd8-8b68-4a89-ba9b-1489791ad68a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:05.752903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ffd8-8b68-4a89-ba9b-1489791ad68a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:05.752903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ffd8-8b68-4a89-ba9b-1489791ad68a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:05.756879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 4/6] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:05.762415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:05.762851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:05.762976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:05.785858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:05.787407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:05.795914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 3/7] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 10 13:56:05.817212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ffd8-8b68-4a89-ba9b-1489791ad68a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:05.829206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ffd8-8b68-4a89-ba9b-1489791ad68a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:05.833416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:05.834258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:05.834632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:05.835127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:05.844217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:05.846034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744876d7-4fe9-4380-bab6-51746db436b2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:05.855337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 5/8] 10.222.0.58 () {58 vars in 1324 bytes} [Tue May 10 13:56:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 13:56:05.862274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:05.871368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcca37-5282-43c1-be12-ab71412adda1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:05.876350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:05.877215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:05.877502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:05.877782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:05.879235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:05.881069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d483475-4de6-4737-b3ac-5a087209aa19 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:05.882710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 4/9] 10.222.0.58 () {56 vars in 1198 bytes} [Tue May 10 13:56:05 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:08.160356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dce4c-93ad-4693-9d1d-733c72c281c4 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.160896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dce4c-93ad-4693-9d1d-733c72c281c4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.160983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dce4c-93ad-4693-9d1d-733c72c281c4 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.162376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 6/10] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:08.167862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588932e5-ed76-4371-953b-76de1438d484 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.168246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588932e5-ed76-4371-953b-76de1438d484 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.168423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588932e5-ed76-4371-953b-76de1438d484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.251694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec3e6-4639-4881-9219-b0a0221f0667 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.251978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec3e6-4639-4881-9219-b0a0221f0667 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.252258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ec3e6-4639-4881-9219-b0a0221f0667 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.253636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 7/11] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:08.259685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.259970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.260295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.297294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-588932e5-ed76-4371-953b-76de1438d484 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:08.298571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-588932e5-ed76-4371-953b-76de1438d484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:08.302076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG passlib.handlers.bcrypt [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80413) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 10 13:56:08.302525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG passlib.handlers.bcrypt [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80413) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 10 13:56:08.349559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:08.351204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:08.377745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 5/12] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:08.401408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588932e5-ed76-4371-953b-76de1438d484 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.452050 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 8/13] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:08.473851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.501264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588932e5-ed76-4371-953b-76de1438d484 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:08.534496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:08.535301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.535583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.535859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.547322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:08.558845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:08.559764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:08.560689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 6/14] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:08.566500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.573327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a37743-8b80-48e3-b10c-2b6c77c73d32 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:08.607144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:08.608676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.608965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.609247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.665681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf327845-4d2f-48f9-a81f-abe13ef57871 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:08.675702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:08.677323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:08.699229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:08.699902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.700183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.700453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.701216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:08.702189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:08.703643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:08.715559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 7/15] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:08.721707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.775991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 9/16] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 13:56:08.781809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.819908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484dab7-815f-41f4-8f1d-fbb072a954f6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:08.854956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:08.855841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.856129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.856413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.857699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:08.859209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:08.860863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 8/17] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:08.866159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:08.883042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832255c-bc7a-4682-aded-96f5e97070e4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:08.915681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:08.916519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:08.916807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:08.917086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:08.967547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929addd-5026-48e0-8e56-de973b4b8e6e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:09.011542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:09.012550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:09.012838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:09.013119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:09.015830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:09.017341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:09.019413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:09.025471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:09.049314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:09.050849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:56:09.072594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:09.073522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:09.074391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:56:09.074391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:56:09.074391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:56:09.074391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:09.074391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:09.080808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:09.125598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6b0f-81ba-4abd-a3f8-48c816c2cce5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:09.160174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:09.160831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:09.161112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:09.161381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:09.162243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.domain.id=default, user.id=462840e7ac324edfaa49acd55da38f49) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:09.162551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:09.164005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757166c5-5ba2-4904-b0f8-e6db8daaa2c6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:09.170764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1322 bytes} [Tue May 10 13:56:09 2022] GET /identity/v3/role_assignments?user.id=462840e7ac324edfaa49acd55da38f49&scope.domain.id=default&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:09.180656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea6d1a-c321-4e27-996b-859d08e9721e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:09.219299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:09.219900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:09.220191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:09.220469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:09.221218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:09.221532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:09.222987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec79ba-2572-4d0e-8d36-bf1da8999811 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:09.243323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:11.879460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7ce7-7852-4c53-ac1b-b4896a1ad71b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:11.880476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7ce7-7852-4c53-ac1b-b4896a1ad71b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:11.880973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7ce7-7852-4c53-ac1b-b4896a1ad71b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:11.882604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:11.895993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:11.896929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:11.897399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:11.924477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:11.926008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:12.024948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:12.059392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc7ce7-7852-4c53-ac1b-b4896a1ad71b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.190993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc7ce7-7852-4c53-ac1b-b4896a1ad71b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:12.234380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:12.235439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.235872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.236323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.252260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-a2ec7011-16c0-4172-8d51-0abf176de8b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:12.260170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:12.261241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:12.262184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:12.268760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 12/24] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:12.274268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.448967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f57ba-4f2c-493c-830a-6a4ff4d00fc3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:12.459868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229bd-8f44-497a-ba2c-287e9fab4e04 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.460516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229bd-8f44-497a-ba2c-287e9fab4e04 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.461026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229bd-8f44-497a-ba2c-287e9fab4e04 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.463051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 13/25] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:12.475515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07142e-613c-49a4-9a0c-e4e900f98f26 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.476163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07142e-613c-49a4-9a0c-e4e900f98f26 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.476645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07142e-613c-49a4-9a0c-e4e900f98f26 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.483489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 14/26] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:12.485322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.485861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.486379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.503975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:12.504693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.504984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.505269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.506056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:12.506401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:12.507514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:12.507836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70a7d5-714b-42f0-8331-e3455f16b31f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:12.509181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:12.536749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 13/27] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:12.538302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.538766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.539261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.567287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:12.568846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:12.582287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 15/28] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:12.583601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.689425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324bcc7-80db-4719-9f41-3dd26ad2eb9e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:12.730699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:12.738535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 14/29] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:12.738535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.739919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.740412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.740888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.742470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:12.744159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:12.746415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 16/30] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:12 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:12.753345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.870887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f61aa4-bc99-4b5f-817a-0b3f45d07635 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:12.907278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:12.908893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.909379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.909867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.911344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:12.913003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:12.915271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 17/31] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:12.916241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:12.938890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe00da-b060-40f8-b36a-3429a9447727 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:12.975463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:12.976591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:12.977092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:12.977563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:12.988820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:12.990630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:56:13.009728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:13.010676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:13.011683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:56:13.011683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:56:13.011683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:56:13.011683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:13.013547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 15/32] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:13.018897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:13.041539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665b4af-fcb6-4d7f-b541-9361d989edbe None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:13.078692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:13.079489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.079692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.079936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.080733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:13.082297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e704a-d6f8-4f69-80c0-ec00caea40b9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.121415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70d42a-c591-43c1-881b-8c5826769fc3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:13.156195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:13.157064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.157358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.157665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] PATH_INFO: `/v3/domains/default/users/462840e7ac324edfaa49acd55da38f49/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.160048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=462840e7ac324edfaa49acd55da38f49, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:13.161502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dffcb-cf33-4b80-a5c4-3b5ef630ab51 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.304885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 18/33] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 10 13:56:12 2022] POST /identity/v3/roles => generated 219 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:13.306664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900126e3-5000-4230-8d80-69c8af9af5f0 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.307103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900126e3-5000-4230-8d80-69c8af9af5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.307548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900126e3-5000-4230-8d80-69c8af9af5f0 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.314082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 19/34] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:13.315327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900126e3-5000-4230-8d80-69c8af9af5f0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:13.360460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 16/35] 10.222.0.58 () {58 vars in 1361 bytes} [Tue May 10 13:56:13 2022] PUT /identity/v3/domains/default/users/462840e7ac324edfaa49acd55da38f49/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:13.365595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9df65-1e77-426a-8fe5-a65b243087b3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.366425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9df65-1e77-426a-8fe5-a65b243087b3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.366828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9df65-1e77-426a-8fe5-a65b243087b3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.373915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 17/36] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:13.380128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.380717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.381179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.402881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:13.405738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:13.494677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900126e3-5000-4230-8d80-69c8af9af5f0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:13.536945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:13.538024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.538575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.539055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.540079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:13.540662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:13.542883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b9a5e-b4cc-4a34-a40d-899365be454f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.571928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 20/37] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:13.577635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.577924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.578234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.593298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 18/38] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:13.594689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:13.606041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:13.608244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:13.687230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 21/39] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:13.688015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:13.729643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4055f00-1c74-460c-981e-9990a1225189 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:13.769225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:13.770365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.770921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.771405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.772333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:13.774185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.817640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117693-2619-4a08-837d-3a7ac5f03174 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:13.851359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:13.851961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.852252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.852533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.853289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:13.854762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808358e3-61c6-457b-81cb-6a1a0a1cf06a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:13.909474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 19/40] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/projects => generated 301 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:13.910399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:13.931782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 22/41] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/roles => generated 225 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:13.934067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118495-9fc7-4e2e-9ed0-43d683b9f614 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.934642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118495-9fc7-4e2e-9ed0-43d683b9f614 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.935183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf118495-9fc7-4e2e-9ed0-43d683b9f614 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.937503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 23/42] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:13.947977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:13.948602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:13.958365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:13.991653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:13.993403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:14.048011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134247-de14-41f0-a63b-540dd56e4279 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:14.090306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:14.091166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:14.091558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:14.091899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:14.093182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:14.094808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:14.140223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 24/43] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:14.163719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:14.206182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 20/44] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 10 13:56:13 2022] POST /identity/v3/roles => generated 223 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:14.288369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a90de-9112-4b2e-beab-a592c854acc7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:14.326152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:14.328191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:14.328498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:14.328757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:14.330038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:14.331560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a79f5-9631-4e76-b77e-ad86a5f3d55f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:14.333416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 25/45] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:14.338949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:14.448006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3721b-3e62-4d09-8cc9-7e4a5b823a8f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:14.485467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:14.485467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:14.485467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:14.485855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:14.486429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:14.487912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4e68-8ef5-47af-a2dd-be9b5cd53b87 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:14.593625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 21/46] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:56:14 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:15.832267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c00b7-b73d-49f4-8586-8391db82ffe0 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:15.832769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c00b7-b73d-49f4-8586-8391db82ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:15.832850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c00b7-b73d-49f4-8586-8391db82ffe0 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:15.834239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 26/47] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:15.839236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:15.839655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:15.839806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:15.856645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:15.857832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:15.922186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 22/48] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:15.950188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c00b7-b73d-49f4-8586-8391db82ffe0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:16.049376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c00b7-b73d-49f4-8586-8391db82ffe0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:16.082018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:16.082831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.083115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.083388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.093049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:16.100496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:16.101335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:16.102092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 27/49] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:16.107386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:16.200534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7fdd4-59ed-4b21-9fef-a6402dbe5633 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:16.234038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:16.234723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.235005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.235278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.236029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:16.236333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:16.237692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:16.245496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 23/50] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:16.251795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:16.348445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556b0fe-99c6-4175-abfc-aa849af30c18 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:16.380318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:16.381194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.381479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.381758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.382967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:16.384421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:16.386048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 28/51] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:16 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:16.391994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:16.487400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31477-e931-4a38-a4b0-dd698b12da61 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:16.519812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:16.520615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.520903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.521186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.522406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:16.523843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bdfbd-ad85-4852-8e2d-aca50ba80f74 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:16.525472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 24/52] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:16.531809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:16.627975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d734a23-1285-406b-9f1c-06c5325e0713 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:16.660274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:16.660927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.661212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.661528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.662353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.domain.id=default, user.id=462840e7ac324edfaa49acd55da38f49) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:16.662667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:16.664102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197d4d-6a04-48f3-9841-a6cebf905fbf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:16.669928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 29/53] 10.222.0.58 () {56 vars in 1322 bytes} [Tue May 10 13:56:16 2022] GET /identity/v3/role_assignments?user.id=462840e7ac324edfaa49acd55da38f49&scope.domain.id=default&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 530 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:16.892296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72651f55-904f-4bd2-9709-cb7907c6b1d0 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.892780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72651f55-904f-4bd2-9709-cb7907c6b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.892863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72651f55-904f-4bd2-9709-cb7907c6b1d0 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.894292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 25/54] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:16.899857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:16.900314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:16.900436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:16.917887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:16.919105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:16.985100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 30/55] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:17.007764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72651f55-904f-4bd2-9709-cb7907c6b1d0 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:17.106338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72651f55-904f-4bd2-9709-cb7907c6b1d0 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:17.138718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:17.139643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:17.139927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:17.141482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:17.141482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:17.142946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5321b-08c7-42ea-8cb1-cc01b609b074 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:17.144546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 26/56] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:17.150119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:17.254405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce295a46-44cf-4393-baa9-7414c43dfb1a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:17.285142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:17.285788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:17.286071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:17.286383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:17.287094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:17.288520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0ecf7-82d3-4c91-bdaa-a6395260c181 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:17.353544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 31/57] 10.222.0.58 () {60 vars in 1140 bytes} [Tue May 10 13:56:17 2022] POST /identity/v3/projects => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:19.621714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4786db-7a23-453f-a797-669c97a096d9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:19.622260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4786db-7a23-453f-a797-669c97a096d9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:19.622366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4786db-7a23-453f-a797-669c97a096d9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:19.623965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 27/58] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:19.629587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:19.630018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:19.630151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:19.647229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:19.648420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:19.713480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 32/59] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:19.739504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4786db-7a23-453f-a797-669c97a096d9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:19.841902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4786db-7a23-453f-a797-669c97a096d9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:19.875964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:19.876752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:19.877038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:19.877317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:19.878511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:19.880001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729747f-13ff-4de1-83cf-90bc531ba733 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:19.881597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 28/60] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:19.887087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:19.979825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9bb02-368c-40d5-a6bf-b515863e72ad None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:20.012921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:20.013590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:20.013877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:20.014186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:20.015115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:20.016709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a882c3-156e-45bd-9099-d6560637f23e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:20.119007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 33/61] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:19 2022] POST /identity/v3/users => generated 275 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:23.615030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae612-f829-479c-81fa-d4a8925cd0c2 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:23.615548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae612-f829-479c-81fa-d4a8925cd0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:23.615620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae612-f829-479c-81fa-d4a8925cd0c2 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:23.617272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 29/62] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:23.622517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:23.622966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:23.623146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:23.651561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:23.652824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:23.740812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 34/63] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:23.782102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bae612-f829-479c-81fa-d4a8925cd0c2 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:23.906820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bae612-f829-479c-81fa-d4a8925cd0c2 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:23.954876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:23.955701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:23.955991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:23.956270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:23.969200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-bc3db143-ad95-44d2-82cf-557d8af8f35c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:23.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:23.979833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:23.980792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:23.982805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 30/64] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:23.989403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.005713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb784-c8ae-44a8-a8b5-43878ad04843 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.006406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb784-c8ae-44a8-a8b5-43878ad04843 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.006875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eb784-c8ae-44a8-a8b5-43878ad04843 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.008534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 31/65] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:24.015560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbee4e-b2e7-417b-bdac-e67223343acb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.016129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbee4e-b2e7-417b-bdac-e67223343acb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.016589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbee4e-b2e7-417b-bdac-e67223343acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.055917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c1bbee4e-b2e7-417b-bdac-e67223343acb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:24.059617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c1bbee4e-b2e7-417b-bdac-e67223343acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:24.145272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 32/66] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:24.147075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd166ff-2c21-4297-96b3-8dd6e538288a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.147552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd166ff-2c21-4297-96b3-8dd6e538288a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.148030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd166ff-2c21-4297-96b3-8dd6e538288a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.149762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 33/67] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:24.156101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd958f-6e33-42a5-9fe5-755904927939 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.159707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.159944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.160234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.188101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:24.190485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:24.196890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.197569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.198024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.198476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.199410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:24.199864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.201418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.211084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 35/68] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:24.212131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.272231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 34/69] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:24.272231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.335597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274491-0a2a-4844-8c85-465008c84892 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.369202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.370124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.370424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.370708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.381711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10b4e-2d29-4a0a-bae3-40ddb256b6ac None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.382144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b9063371-3911-4bae-a8fe-5fe32020f5d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:24.390548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:24.391456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:24.392300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 36/70] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:24.395604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5c49-d3b2-4810-acac-496e96b05f77 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.396392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5c49-d3b2-4810-acac-496e96b05f77 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.396849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5c49-d3b2-4810-acac-496e96b05f77 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.398448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 37/71] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:24.399339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e5c49-d3b2-4810-acac-496e96b05f77 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.421730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.423002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.423486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.423950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.440221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.441752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.443498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 35/72] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:24.444278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.512417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e5c49-d3b2-4810-acac-496e96b05f77 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.540170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2213f-7684-49a4-91d1-779c03d7fc2c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.544356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.545170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.546067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.546381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.556452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-85e3ee69-04b3-4927-86d6-a23e43664390 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:24.564891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:24.565738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:24.566759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 38/73] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:24.567574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91434d3-8dd6-42c3-9348-beb0966d3c0e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.567574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91434d3-8dd6-42c3-9348-beb0966d3c0e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.568883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91434d3-8dd6-42c3-9348-beb0966d3c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.574569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.575242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.575533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.575815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.576575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:24.576891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.578309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.584261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f91434d3-8dd6-42c3-9348-beb0966d3c0e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:24.585467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f91434d3-8dd6-42c3-9348-beb0966d3c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:24.586374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 36/74] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:24.588202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.651315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 39/75] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:24.652584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bf32-e8b3-4d51-94d0-1d54c79c9364 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.652946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bf32-e8b3-4d51-94d0-1d54c79c9364 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.653294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67bf32-e8b3-4d51-94d0-1d54c79c9364 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.654705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 40/76] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:24.656049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67bf32-e8b3-4d51-94d0-1d54c79c9364 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.703828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9415a0-be48-451e-a503-0d76ecc7a72d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.737727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.738413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.738703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.738985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.768271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67bf32-e8b3-4d51-94d0-1d54c79c9364 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.802560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.803235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.803522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.803802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.804557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:24.804867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.806281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.814598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 41/77] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:24.814869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.816230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f206e9-7514-43dd-91eb-a714a147e46a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.819134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:24.920773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 37/78] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:24.922405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.922819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.923056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.937008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545feac1-e267-4abf-9514-a1a10a66c7e9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:24.941926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:24.943756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:24.972867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:24.973694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:24.973957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:24.974253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:24.975483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:24.976953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:24.978681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 42/79] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:24.983986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.013842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 38/80] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:25.015824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.087736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830d09-5f1d-43c2-8d91-f6dc96ec2d7c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.119899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.120690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.120974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.121253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.121640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bf0cb-b4fc-431c-bd09-16a099d38bef None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.133568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:25.142218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:25.143342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 43/81] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:25.144343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.159313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.160188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.160471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.160745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.161928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.163421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.165033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 39/82] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.165792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.256313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb5e09-a146-4561-83a5-cae40b6cea65 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.274996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a8a2-bacc-4331-a6c3-0256fd007c01 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.289013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.289879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.290170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.290483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.291671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.293131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.295088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 44/83] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.295910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.311716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.312381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.312662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.312941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.378172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.379663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.399642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef02-5aba-4279-a1de-8b754ce341f9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.437536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.438371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.438662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.438953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.440148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.441615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.443286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 45/84] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.480263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 40/85] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:25.482333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.546292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722407-515c-4ad9-9451-475db0ab50d1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.579559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6113-73d8-4e1b-b63a-82f3b56150b5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.581264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.581959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.582281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.582559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.583344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:25.583652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.585208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.604741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 46/86] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.605614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.617011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.617666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.617949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.618247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.684552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.686136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.716458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3463b9b-3151-4280-bc62-3bb328ceccaa None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.751526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.752178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.752474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.752753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.753599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:25.753914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.755418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.762756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 47/87] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.764109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.813606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 41/88] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:25.813606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:25.894008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab265c-ca7f-4bb1-ae97-ce7d9a17c1f3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.921479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4465c-381a-4064-b887-9f84744f3990 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:25.929645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.930367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.930653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.930933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.931668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.933165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.958833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:25.959968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:25.960253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:25.960531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:25.961739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:25.963219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:25.964940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 42/89] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:25.967207 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.033285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 48/90] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:25 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:26.033285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.072875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7ca1-339c-407d-8292-d1569f183726 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.113990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.114913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.115197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.115475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.116749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.118225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.119985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 43/91] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:25 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:26.126178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.147154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63172-7713-4c24-a8f8-ae3ae844c160 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.187955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.188848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.189133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.189412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.191304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.193373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.195256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 49/92] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:26.196160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.238288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4dae9-b855-40f5-b446-63a37c1bea7f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.281842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.282891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.283401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.283880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.284958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=462840e7ac324edfaa49acd55da38f49) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:26.285275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.287643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.296386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 44/93] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/role_assignments?user.id=462840e7ac324edfaa49acd55da38f49&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:26.297285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.316160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c02c13-27e8-4bfd-bb2d-04c8f73921e8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.357609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.358785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.359078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.359362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.415850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00994ff3-6c72-47b1-96ed-2b4d48a31874 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.438428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.439943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.458921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.459614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.459904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.460189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.461049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:26.461369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.462897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.470268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 45/94] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=4642f59bea334153a5500859e246a0c6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:26.565993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 50/95] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:26.571999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.703914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f17945-b954-4f45-97c7-3b326fe77667 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.770542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.771425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.771711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.771989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] PATH_INFO: `/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.773214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] RBAC: Authorizing `identity:get_project(project_id=43cbd389e4f8493fb1c56ebe3777ed5b)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.774730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f051d6-1d7b-4b50-8b0f-2b79b75865af None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.776479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 46/96] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:26.782579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:26.903090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd53e3-3db2-44ca-aaee-9016b868206e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:26.945359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:26.946038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:26.946354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:26.946637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:26.947495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=43cbd389e4f8493fb1c56ebe3777ed5b, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:26.947811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:26.949286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc458885-1e9e-4b70-affa-8c699ab404bb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:26.956410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 51/97] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:26 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=43cbd389e4f8493fb1c56ebe3777ed5b&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:29.165174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f582d-c561-48d8-b10f-9a0e20d6eeff None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.165721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f582d-c561-48d8-b10f-9a0e20d6eeff None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.165721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f582d-c561-48d8-b10f-9a0e20d6eeff None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.167185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 47/98] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:29.184330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f2e4-024c-42e0-8a06-2df3a3e75422 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.185205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f2e4-024c-42e0-8a06-2df3a3e75422 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.185643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57f2e4-024c-42e0-8a06-2df3a3e75422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.215327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0c57f2e4-024c-42e0-8a06-2df3a3e75422 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:29.216799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0c57f2e4-024c-42e0-8a06-2df3a3e75422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:29.295963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 52/99] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:29.315001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35aa76-0025-41f2-9920-de7716fceb83 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.315289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35aa76-0025-41f2-9920-de7716fceb83 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.315576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35aa76-0025-41f2-9920-de7716fceb83 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.316950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 48/100] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:29.322456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.322979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.323418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.337538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35aa76-0025-41f2-9920-de7716fceb83 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:29.348611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:29.349833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:29.512846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 53/101] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:29.519147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35aa76-0025-41f2-9920-de7716fceb83 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:29.552149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:29.575401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:29.576639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.577493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.577778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.596648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:29.604552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:29.605536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:29.606335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:29.609717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 49/102] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:29.616012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:29.832697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63abe2e-a2e4-472b-9cd2-817612cbeabe None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:29.868429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68a5d1-2350-46b4-a2d0-2d631489c054 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:29.907121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:29.908618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:29.909547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.910588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.911116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.911596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.912062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.913039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:29.913520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.914804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:29.916664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c13843-5f08-471f-a190-da0ed2ee2177 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:29.932480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 50/103] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:29.934167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8644-6430-4006-bd62-4fd9d4db69f8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.934802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8644-6430-4006-bd62-4fd9d4db69f8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.935218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8644-6430-4006-bd62-4fd9d4db69f8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:29.937035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 51/104] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:29.941870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8644-6430-4006-bd62-4fd9d4db69f8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:29.946376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-687e70c9-46f7-4e5f-aafd-0508a17b3bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:29.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:29.961368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:29.962330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:29.968391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 54/105] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:29.974224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:29.975723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:29.976339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.007290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:30.009383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:30.098294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 55/106] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:30.098475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8644-6430-4006-bd62-4fd9d4db69f8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.103365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.135284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.136534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.136995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.137427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.139269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.141615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ce87b-ded8-4720-b33d-d11305b66999 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.143702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 52/107] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:29 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:30.145273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606c29-a462-4b4a-8df7-7461cd8b3e19 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.145569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606c29-a462-4b4a-8df7-7461cd8b3e19 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.145854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606c29-a462-4b4a-8df7-7461cd8b3e19 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.147780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 53/108] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:30.155876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596c9e-b7e0-4d1c-b59c-d527a700da0e None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.155876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596c9e-b7e0-4d1c-b59c-d527a700da0e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.155876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596c9e-b7e0-4d1c-b59c-d527a700da0e None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.155876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 54/109] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:30.155876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7596c9e-b7e0-4d1c-b59c-d527a700da0e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.227684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1150e-65e0-4539-a917-457a6c39220c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.262659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.264156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.264156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.264156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.264945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7596c9e-b7e0-4d1c-b59c-d527a700da0e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.265614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:30.265934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.267517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.282105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 56/110] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:30.282105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.300630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.301457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.301742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.302020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.318486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-cd8afb59-cd9c-4d47-900d-f035f7e60698 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:30.326654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:30.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:30.328941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:30.330738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 55/111] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:30.332843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6780-7082-4c41-bbf0-5faea8f4e32e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.333495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6780-7082-4c41-bbf0-5faea8f4e32e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.333787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6780-7082-4c41-bbf0-5faea8f4e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.351956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c7da6780-7082-4c41-bbf0-5faea8f4e32e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:30.353625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c7da6780-7082-4c41-bbf0-5faea8f4e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:30.387071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626a95-825f-4f29-803c-3d1b9dcceb02 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.426919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.427418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 56/112] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:30.427787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.428075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.428353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.428716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.429026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.429288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.429687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.431205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.438873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 57/113] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:30.438873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.447010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:30.448209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:30.516467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 57/114] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:30.517332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.547921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69417fc-38e7-4515-85ae-953ddb0894f2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.581137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.582002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.582316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.582595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.583789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.585250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05feb4-1f33-4fe4-bb8a-09e6076ba5f1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.586953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 58/115] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:30.588120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e255fa-16d0-4616-883a-8ac743c371e4 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.588403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e255fa-16d0-4616-883a-8ac743c371e4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.588681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e255fa-16d0-4616-883a-8ac743c371e4 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.589895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 59/116] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:30.590596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e255fa-16d0-4616-883a-8ac743c371e4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.627016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181b988-564d-402a-8c2d-e35aa97b3cc4 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.660965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.661723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.661937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.662242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.663004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:30.663313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.664726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.673732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 58/117] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:30.675858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.709005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e255fa-16d0-4616-883a-8ac743c371e4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.748353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.749544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.749859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.750154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/deac13a59faa44bab73388e308e1cb8f/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.753113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, user_id=deac13a59faa44bab73388e308e1cb8f, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:30.754618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:30.788746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db816505-749f-464a-a75e-aa6a59a639fb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.827181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:30.828238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:30.828527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:30.828806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:30.838965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:30.839197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 60/118] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:30 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/deac13a59faa44bab73388e308e1cb8f/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:30.847938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:30.854684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 59/119] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:30.855597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.856288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:30.966946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990deab-56fb-42ca-a282-0389bd7389c2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:30.996100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83cca-1522-4818-9a58-06e26a3f042b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.009640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.010494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.010785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.011068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.012360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.013858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce700c6-789d-43d3-b210-d15c9d023cb5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.016385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 61/120] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:31.017664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.017954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.018255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.038265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:31.039486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:31.055019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.057122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.057548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.057845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.059372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.061065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.063995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 60/121] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:30 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:31.065296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.133368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 62/122] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:31.134551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.183976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adde4c-b4e4-43b4-b578-5d1137b61d0e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.221841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.222681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.222980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.223274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.225060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.226582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.232648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 61/123] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:31.232648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.243969 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93fdc2-7092-48fa-ada3-34e8a6f83f3e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.283825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.284502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.284783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.285057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.285877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:31.286189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.287665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.301055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 63/124] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:31.301055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.341045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be849e1-4c15-40f1-94ee-3de07031805b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.382494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.383467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.383971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.384439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.385195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.386676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.416145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132d8c-d9ba-4411-9ab2-12a64dfc63a2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.458489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.459880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.459880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.460089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/462840e7ac324edfaa49acd55da38f49/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.462039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, user_id=462840e7ac324edfaa49acd55da38f49, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.463545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.503801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 62/125] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:56:31 2022] POST /identity/v3/projects => generated 302 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:31.515929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.623319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 64/126] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:31 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/462840e7ac324edfaa49acd55da38f49/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:31.625457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.653768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78252-21f0-4e1e-b090-447ab373b3ad None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.752213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.753597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.754106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.754400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.785891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:31.794324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:31.795310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:31.796848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:31.803648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 63/127] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:31.803648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.845438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4379-9c46-41ae-ac5f-5ba38d4414b0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:31.894613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:31.896029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:31.896322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:31.896606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:31.898144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:31.900835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:31.903065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 65/128] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:31.903629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:31.936145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0408-49dd-478e-b11e-3aeb301039b8 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.035079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.035918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.036217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.036501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] PATH_INFO: `/v3/users/5a0c3afc2c1242d4a2542064acb0179c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.040181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514a22-4b86-4438-96ae-b24d977a27a9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.071036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c3afc2c1242d4a2542064acb0179c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.072559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.074616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 64/129] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/users/5a0c3afc2c1242d4a2542064acb0179c => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:32.075209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.086986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.088104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.088401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.088683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.089448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:32.089758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.091162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.101930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 66/130] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:32.102639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.215594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba69d6-60b0-4b40-b23d-a6a216cd4cde None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.251930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.252691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.252871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.253158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.275549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271863f-4da0-4445-b709-9abc0f492f61 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.324170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.325555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.348811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.349695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.349982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.350285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/deac13a59faa44bab73388e308e1cb8f/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.360503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, user_id=deac13a59faa44bab73388e308e1cb8f, role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.362094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.450642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 67/131] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:32.453409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.471111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 65/132] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:32 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/deac13a59faa44bab73388e308e1cb8f/roles/4642f59bea334153a5500859e246a0c6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:32.471984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.568016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6522ec-fc6c-4cf5-8610-55959314ddf1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.605803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.606650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.606929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.607203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.609391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.611162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.612913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 68/133] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:32 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:32.618460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.712590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135253-e3ce-47a9-b351-814eb7e77124 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.739068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d605d7-e141-418c-93b8-1fdbf4655c03 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.784642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.785537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.785824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.786102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] PATH_INFO: `/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.787416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] RBAC: Authorizing `identity:get_project(project_id=43cbd389e4f8493fb1c56ebe3777ed5b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.788912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.791039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 69/134] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:32 2022] GET /identity/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:32.797814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.801038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.802282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.802793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.803269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.811180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.813702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:32.816226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 66/135] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:32 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:32.822339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:32.908005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd7dc-0b46-49e6-8d84-9787f0edde41 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:32.944789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:32.945687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:32.945973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:32.946268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] PATH_INFO: `/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b/users/deac13a59faa44bab73388e308e1cb8f/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:32.948246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=43cbd389e4f8493fb1c56ebe3777ed5b, user_id=deac13a59faa44bab73388e308e1cb8f, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:32.949712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13b631-f865-4f59-8113-b465beee2dd3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:33.003385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e951a-5805-4353-b8cc-d36bdb493d3a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:33.035490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 70/136] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:32 2022] PUT /identity/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b/users/deac13a59faa44bab73388e308e1cb8f/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:33.060162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:33.060886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:33.061170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:33.062418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:33.067488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=5a0c3afc2c1242d4a2542064acb0179c) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:33.067488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:33.067488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3158f20-69ed-4124-8bbd-c1e59ea4893a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:33.072125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 67/137] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:32 2022] GET /identity/v3/role_assignments?user.id=5a0c3afc2c1242d4a2542064acb0179c&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:33.665444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90378d-8eb3-4089-be12-dbaaac4136c2 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:33.665992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90378d-8eb3-4089-be12-dbaaac4136c2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:33.665992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90378d-8eb3-4089-be12-dbaaac4136c2 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:33.667638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 71/138] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:33.674243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:33.674715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:33.674826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:33.693992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:33.695219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:33.799026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 68/139] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:33.864009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90378d-8eb3-4089-be12-dbaaac4136c2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:33.991222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90378d-8eb3-4089-be12-dbaaac4136c2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:34.027212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:34.028018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.028303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.028588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.040909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:34.049900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:34.050850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:34.051754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 72/140] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:34.058734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:34.219488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6dbae-964c-4288-a5cc-86036fe2b587 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:34.264323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:34.265579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.265867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.266147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.266926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:34.267242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:34.268628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:34.281525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 69/141] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:34.287419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:34.427328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498e452-4edd-4415-8564-0db17f7245a1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:34.462636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:34.463577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.463861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.464138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.465358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:34.466852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa58ac1-65fd-4a38-ae63-64b40150a1be None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:34.469992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 73/142] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:34 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:34.479396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:34.645593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebed46-1207-4870-b4ed-266ae038455f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:34.687579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:34.688431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.688723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.689005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.713464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ddfc-817b-42a5-b289-da843fc3c8d1 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.714187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ddfc-817b-42a5-b289-da843fc3c8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.714187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ddfc-817b-42a5-b289-da843fc3c8d1 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.715426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 74/143] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:34.727749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:34.728274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:34.728754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:34.765667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:34.772516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:34.851303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:34.853142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:34.925700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 75/144] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:34.981687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.012469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 70/145] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:35.013985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.149407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bf3b3-0c09-4efb-96a2-c9fa9eca26fa None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:35.193180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:35.193982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.194291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.194574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.199014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:35.200497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:35.202333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 71/146] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:35 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:35.208295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.236725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7437dc4-acbe-448f-a4eb-abd60935d154 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:35.352807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d16247-ddea-4322-ac4f-cf05e957b6fb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:35.364966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:35.366078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.366400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.366697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.371446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:35.373332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e3d3a-26c6-41a1-9889-00639ee4488f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:35.375885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 76/147] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:35.384218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb9ab-1144-42c1-a92f-c04d8dc328e7 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.384662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb9ab-1144-42c1-a92f-c04d8dc328e7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.385029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb9ab-1144-42c1-a92f-c04d8dc328e7 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.386970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 77/148] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:35.387532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fb9ab-1144-42c1-a92f-c04d8dc328e7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.397905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:35.399036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.399515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.399981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.401011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:35.401551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:35.404225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3f5b-124c-45d0-8896-d6d43b8f1520 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:35.417428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 72/149] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:35 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:35.418604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.418895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.419176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.462169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:35.463343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:35.548161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fb9ab-1144-42c1-a92f-c04d8dc328e7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:35.577909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 73/150] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:35.601963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:35.602710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.602994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.603270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.604011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:35.605548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:35.621406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.754282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e5966-ae4d-455d-b908-70b17de8db9c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:35.765030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 78/151] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:35 2022] POST /identity/v3/users => generated 283 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:35.802166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:35.803020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.803317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.803607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.820349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b49edfbf-023a-4d93-925f-46cbcf363673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:35.829782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:35.830756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:35.831509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:35.832280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 74/152] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:35.837960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:35.966794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d0b1-c681-44c7-82b3-c8ddeaefb7c9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.967179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d0b1-c681-44c7-82b3-c8ddeaefb7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.967346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411d0b1-c681-44c7-82b3-c8ddeaefb7c9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.969596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 75/153] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:35.975544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:35.975845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:35.976035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:35.980462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef3f9-2c76-41b9-ac40-f1f716b508f0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.002142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:36.005025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:36.024144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.025140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.025615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.026395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.028096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:36.028623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.030279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.039520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 79/154] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:36.045410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.089323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 76/155] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:36.117772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.169123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0014ab2-30de-48da-a859-89abc07504fd None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.207718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.208656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.208985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.209625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] PATH_INFO: `/v3/users/462840e7ac324edfaa49acd55da38f49` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.210652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=462840e7ac324edfaa49acd55da38f49)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.212197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.218279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 80/156] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/users/462840e7ac324edfaa49acd55da38f49 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:36.220391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.230117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fac44-b9b6-400f-9df2-f73a75db2006 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.268292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.269311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.269704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.270098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.282491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:36.290982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:36.291904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:36.292695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 77/157] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:36.300014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.336720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e3278-f8f1-47d5-911c-d06b3e94c8f0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.378185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.379229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.379714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.380198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.413006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2b743-d1b8-4a94-931f-b7d3a68dce38 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.453823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.454557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.454901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.455186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.455958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:36.456270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.458258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.460545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.462274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.467079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 78/158] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:36.473333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.576535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 81/159] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 13:56:36.583278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.593378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90145472-be0b-44d9-9f3c-033182c4424c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.631728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.632696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.632990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.633277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.635188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.636303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256f6c4-8f8b-4a65-a836-63077ad014b0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.638083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 79/160] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:36.644781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678106-331c-4f1a-aba4-aa714f889b83 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.645075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678106-331c-4f1a-aba4-aa714f889b83 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.645485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678106-331c-4f1a-aba4-aa714f889b83 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.647200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 80/161] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:36.651314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678106-331c-4f1a-aba4-aa714f889b83 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.716420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c469c32-5964-40f0-b41c-a13e7abcc9c9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.768475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647c5e8-4beb-4f5a-a538-9bd42d7e0031 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:36.788057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 82/162] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:36.788507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.788507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.788507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.795279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:36.797025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:36.805318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678106-331c-4f1a-aba4-aa714f889b83 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:36.871616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:36.872382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:36.873046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:36.873340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:36.887159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 83/163] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:36.887979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:36.970606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:36.972107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4004e-b7d4-4380-b584-99c9886e1cc3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:37.027718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa164f0-5633-4841-9117-3f9429353aa0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.079257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.080093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.080392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.080673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.082272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=462840e7ac324edfaa49acd55da38f49) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:37.082597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:37.084549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657623-93f9-4dad-bec1-f2446eb09591 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:37.092342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 84/164] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/role_assignments?user.id=462840e7ac324edfaa49acd55da38f49&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:37.094764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e053e8-6053-40f5-8c52-a816ee84b240 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.095179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e053e8-6053-40f5-8c52-a816ee84b240 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.095463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e053e8-6053-40f5-8c52-a816ee84b240 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.097046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 85/165] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:37.103090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e053e8-6053-40f5-8c52-a816ee84b240 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.107892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 81/166] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:37.109444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.109744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.110072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.132931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:37.134332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:37.251322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e053e8-6053-40f5-8c52-a816ee84b240 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.278234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 82/167] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:37.278234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.304003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.305150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.305640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.305923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.327863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:37.335874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:37.336817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:37.337634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 86/168] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:37.338309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.451797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad9c0-0ed1-42d9-baad-3b91951b0d8a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.535310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.536207 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.536494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.536772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.538293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:37.540540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:37.542388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 83/169] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:37.546450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.567979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e3005-3ee6-4db8-8205-15d16b3ae93f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.656943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.657771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.658055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.658375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.678841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:37.686801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:37.687902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:37.688704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 87/170] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:37.689535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.714851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fdc09-640e-43a0-9ceb-3a55407a322f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.769909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.770628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.770883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.771165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.771912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:37.772216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:37.773629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:37.788016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 84/171] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:37.788888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.806941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2717e-0fc6-4700-93d1-bc0b6d2faac9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.845214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:37.845886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:37.846172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:37.846478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:37.847338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:37.847677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:37.849113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:37.855630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 88/172] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=4642f59bea334153a5500859e246a0c6 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:37.856581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:37.957195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca754a-ec74-41b4-85e5-54c76a3e3474 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:37.974421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b234fba-8a09-44d2-b959-e00f39ec7c5e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:38.022246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.023564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.024192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.024750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.025116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.026020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.026335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.026615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] PATH_INFO: `/v3/users/5a0c3afc2c1242d4a2542064acb0179c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.026881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:38.027498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.034253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.034457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c3afc2c1242d4a2542064acb0179c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.034457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.035488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 89/173] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/users/5a0c3afc2c1242d4a2542064acb0179c => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:38.042220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:38.049204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 85/174] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:38.055180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:38.239301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b17-bfbc-4d0a-a894-ce073d4fb030 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:38.242642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6123-d8b9-4ad6-9a2e-67ebc1e56a2a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:38.296386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.297761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.298268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.298709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.300671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.302693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.303166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.303578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.303863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.304139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] PATH_INFO: `/v3/users/deac13a59faa44bab73388e308e1cb8f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.305321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 90/175] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:38 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:38.311047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=deac13a59faa44bab73388e308e1cb8f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.313237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.314540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:38.315002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 86/176] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:38 2022] GET /identity/v3/users/deac13a59faa44bab73388e308e1cb8f => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:38.327885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:38.521059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc9b4-5c57-442d-9807-dd76a3e6cd0d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:38.536009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f4d3-a9a7-48b6-a01b-bcbd6c924cb0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:38.601984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.603054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.603341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.603626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/5a0c3afc2c1242d4a2542064acb0179c/roles/eea647ad81e945d6a7b6f7ca348970bf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.606441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, user_id=5a0c3afc2c1242d4a2542064acb0179c, role_id=eea647ad81e945d6a7b6f7ca348970bf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.608428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.614246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:38.614893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:38.615260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:38.615546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:38.692927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 91/177] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:38 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/users/5a0c3afc2c1242d4a2542064acb0179c/roles/eea647ad81e945d6a7b6f7ca348970bf => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:56:38.706718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:38.714231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:38.919945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 87/178] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:38.926522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:39.053086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac5e22-08d3-428d-a2f2-d6483fc99d82 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:39.104600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:39.105484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.105770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.106046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] PATH_INFO: `/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.107353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] RBAC: Authorizing `identity:get_project(project_id=43cbd389e4f8493fb1c56ebe3777ed5b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:39.109706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9c2fd-74b0-4586-b5ec-e872a684ac74 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:39.111783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 92/179] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:38 2022] GET /identity/v3/projects/43cbd389e4f8493fb1c56ebe3777ed5b => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:39.118092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:39.244365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64437b-f22b-43eb-bffd-a8e879d21456 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:39.284527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:39.285594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.286079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.286584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.287625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=43cbd389e4f8493fb1c56ebe3777ed5b, user.id=deac13a59faa44bab73388e308e1cb8f) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:39.288160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:39.289941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d36521-2b12-4b78-9bf9-c4d8d69073ff None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:39.298381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 88/180] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:39 2022] GET /identity/v3/role_assignments?user.id=deac13a59faa44bab73388e308e1cb8f&scope.project.id=43cbd389e4f8493fb1c56ebe3777ed5b&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:39.621761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1ee9-70e9-4cf1-90a2-0860d91c9d95 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.622100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1ee9-70e9-4cf1-90a2-0860d91c9d95 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.622253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f1ee9-70e9-4cf1-90a2-0860d91c9d95 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.623661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 93/181] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:39.630908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.631539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.632020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.651625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:39.653221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:39.663305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b99d0-3225-4d88-a884-18413cc4776f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.663608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b99d0-3225-4d88-a884-18413cc4776f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.663881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b99d0-3225-4d88-a884-18413cc4776f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.665656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 94/182] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:39.673153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.673459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.673725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.695097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:39.696809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:39.754183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 89/183] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:39.783147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:39.803249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 95/184] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:39.829378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:39.912161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249577b2-e786-4ad5-9fec-35e394a65aca None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:39.941142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a653a-b2dd-44d1-9b51-335ceea05003 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:39.947480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:39.948300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.948599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.948881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.961781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e82887d8-ad32-4bbe-b45d-e16717fa7ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:39.970771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:39.971760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:39.972677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 90/185] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:39.973595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1648c-5f39-4c57-a32c-377be82400cc None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.973887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1648c-5f39-4c57-a32c-377be82400cc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.974167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1648c-5f39-4c57-a32c-377be82400cc None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:39.975656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 91/186] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:39.980832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1648c-5f39-4c57-a32c-377be82400cc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:39.987407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:39.988219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:39.988504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:39.988781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.001859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-090d66df-a251-4bff-a305-5b0574476f8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:40.009720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:40.010835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:40.011760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 96/187] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:40.012394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.013087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.013087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.032853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:40.034084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:40.102054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1648c-5f39-4c57-a32c-377be82400cc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.110252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 97/188] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:40.110252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.147561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.148887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.149181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.149466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.150764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:40.151220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.152757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.173739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 92/189] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:40.173739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.225356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af45e1-6963-467e-b87c-259c690ab991 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.265184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.265874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.266157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.266462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.267699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:40.268023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.269812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.286177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 98/190] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:40.286177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.292564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddeb4e-f40e-4bce-9181-531caecf4393 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.327925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.328735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.329027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.329304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.330566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.332039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.333695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 93/191] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:40.336338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.399168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5902-8d3e-4e57-9a8c-5a705fc40bb2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.436325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.437680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.437960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.438260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.452125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.452425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61dd18-a6c8-4b59-b67a-5366c4b00f59 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.454396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.460788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 99/192] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:40.460788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.490302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.491526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.491986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.492457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.493855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.495513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.497364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 94/193] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:40.498362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.585150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194426-b17c-4c3c-bf7d-fde8cefc9474 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.613401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e431-9908-4cdc-a2df-a98faceb3ae0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.637724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.638442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.638725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.639066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.640598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.642108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.659736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.660394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.660673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.660945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.734431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:40.736168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:40.782859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 100/194] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:40 2022] POST /identity/v3/users => generated 297 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:40.783900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.849488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 95/195] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:40.855448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:40.918691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c5a69-b5a3-4c5c-8215-65006a7ac817 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:40.973501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:40.974118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:40.974541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:40.974884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:40.994528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0913b-a6a2-41ce-b372-7748548d2112 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:41.039852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:41.040728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.041010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.041282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.042562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:41.044054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:41.045817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 96/196] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:41.050308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:41.051964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:41.052617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:41.179255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 101/197] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:41.186324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:41.208225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8225b0-71a2-4f26-a6cb-11d4e7f862eb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:41.251478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:41.252057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.252343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.252620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.253474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb41f77c34c24422beb85b825bf2ddb5) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:41.253791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:41.255295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:41.261622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 97/198] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:41 2022] GET /identity/v3/role_assignments?user.id=cb41f77c34c24422beb85b825bf2ddb5&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=4642f59bea334153a5500859e246a0c6 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:41.316834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43bd5-b491-478f-a4e8-aea57b34bbc0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:41.375986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:41.376918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.377165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.377441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.378952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:41.380480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2450-d6cd-46e5-9038-c3fb7ac30559 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:41.382720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 102/199] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:41 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:41.388936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:41.537231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc539a-d019-48a0-a039-4032d7acc8e5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:41.582631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:41.583683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.584202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.584691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.585724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb41f77c34c24422beb85b825bf2ddb5) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:41.586236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:41.587910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45699456-752b-483a-a7d1-d2010dcb8002 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:41.598929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 98/200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:41 2022] GET /identity/v3/role_assignments?user.id=cb41f77c34c24422beb85b825bf2ddb5&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:41.717504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503fda-6967-4942-ab05-3487ead516ff None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.719138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503fda-6967-4942-ab05-3487ead516ff None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.719614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26503fda-6967-4942-ab05-3487ead516ff None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.721363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 103/201] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:41.727535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:41.728090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:41.728571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:41.759486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:41.761123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:41.864402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 99/202] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:41.889686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503fda-6967-4942-ab05-3487ead516ff None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:42.035432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26503fda-6967-4942-ab05-3487ead516ff None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:42.073352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:42.074589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:42.075118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:42.075612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:42.087944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:42.097118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:42.098048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:42.098882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:42.101680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 104/203] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:42.107442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:42.263461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02612501-0324-4cc9-bb88-b355feeb78ec None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:42.314587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:42.315278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:42.315563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:42.315848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:42.316619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:42.316928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:42.318369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:42.329852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 100/204] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:42.336149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:42.503528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41599a1a-6dae-4866-aecd-fb6dd3003056 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:42.555145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:42.555945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:42.556225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:42.556499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] PATH_INFO: `/v3/users/5a0c3afc2c1242d4a2542064acb0179c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:42.557715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c3afc2c1242d4a2542064acb0179c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:42.563514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:42.565254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 105/205] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:42 2022] GET /identity/v3/users/5a0c3afc2c1242d4a2542064acb0179c => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:42.571378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:42.766384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a3e1-66b6-4dbf-b879-99951f175e91 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:42.803042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:42.804105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:42.804600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:42.805093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:42.901783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:42.907734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:43.012680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 101/206] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:43.019337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:43.135098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8806-7413-443a-8267-3eb6c5c3d817 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:43.170679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:43.172262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:43.172722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:43.173025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:43.174264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:43.175905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439da64-4a67-45ce-a0f8-8d416272f53f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:43.177617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 106/207] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:43 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:43.183902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:43.348100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f305-370f-4940-a00b-f564e1ab58b7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:43.405315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:43.406716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:43.407014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:43.407398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:43.408255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, scope.project.id=381a803ca3e2428db7752680e6ae1d58, user.id=5a0c3afc2c1242d4a2542064acb0179c) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:43.408571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:43.410035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f046f1b-f236-43fe-acb9-3c3c58a868ef None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:43.419151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 102/208] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:43 2022] GET /identity/v3/role_assignments?user.id=5a0c3afc2c1242d4a2542064acb0179c&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:43.700890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c40-bca5-4636-ac12-a17aad69540e None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:43.701278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c40-bca5-4636-ac12-a17aad69540e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:43.701474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c40-bca5-4636-ac12-a17aad69540e None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:43.704579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 107/209] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:43.710470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:43.710754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:43.711032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:43.744563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:43.745816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:43.835840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 103/210] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:43.861116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673c40-bca5-4636-ac12-a17aad69540e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:43.983461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673c40-bca5-4636-ac12-a17aad69540e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.022863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.023671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.023954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.024227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.036735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-371433e2-5e0b-4001-b9f9-0237d92a4d82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:44.044539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:44.045506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:44.046932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 108/211] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:44.052813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.067197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf15c9-a7b3-4344-ba90-aaddc4ec4525 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.067531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf15c9-a7b3-4344-ba90-aaddc4ec4525 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.067749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf15c9-a7b3-4344-ba90-aaddc4ec4525 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.069215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 109/212] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:44.076191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.076490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.076733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.107049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:44.108560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:44.176240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd81b68-a37a-4277-a8e3-839ff4fc48f7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.188444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 110/213] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:44.212394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.213251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.213534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.213813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.214286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.214612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:44.214923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:44.216323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:44.226082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 104/214] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:44.232085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.344333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38908a-6594-4053-8e11-5218bb6d3d42 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.347147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746dcfc-e300-408d-9690-166427bc4f0f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.382344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.383230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.383521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.383798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] PATH_INFO: `/v3/users/226bc788b84346638e5bcbeb38947aae` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.387304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.388074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.388353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.388624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.389817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:44.391690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95bd69-3579-4db3-9ec8-2d8914e65e1c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:44.394246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 111/215] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:44.394774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d522-41c2-49b0-b7f9-97e5c8f31e47 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.395050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d522-41c2-49b0-b7f9-97e5c8f31e47 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.395321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d522-41c2-49b0-b7f9-97e5c8f31e47 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.398243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 112/216] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:44.398783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d522-41c2-49b0-b7f9-97e5c8f31e47 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.406796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] RBAC: Authorizing `identity:get_user(user_id=226bc788b84346638e5bcbeb38947aae)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:44.409103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e445c-ff43-4d70-852d-e9528c1bfde5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:44.415703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 105/217] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/users/226bc788b84346638e5bcbeb38947aae => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:44.415703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.415703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.415703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.432893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:44.434102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:44.525191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d522-41c2-49b0-b7f9-97e5c8f31e47 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.528654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 106/218] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:44.529239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.566227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.567080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.567695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.567927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.572600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:44.572600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:44.642955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eecde6-3183-42e0-8d63-4ac256ec2098 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.677358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.678083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.678396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.678675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.705169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 113/219] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:44 2022] POST /identity/v3/users => generated 297 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:56:44.706654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.760879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:44.762424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e1e1-b7b4-4c4e-9649-e26631e1e5b3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:44.829828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f706d-24ee-4a01-99b3-52371217bf67 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:44.869062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:44.869877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.870164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.870473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.882678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 107/220] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:44.884079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718ff3e-5f44-4cdd-b37f-46d82b9210aa None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.894340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718ff3e-5f44-4cdd-b37f-46d82b9210aa None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.894340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718ff3e-5f44-4cdd-b37f-46d82b9210aa None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.894340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 108/221] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:44.894651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:44.894729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718ff3e-5f44-4cdd-b37f-46d82b9210aa None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b5f197b8-2cde-4b85-aabb-95a6773f95b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:44.896585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:44.897495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:44.898359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 114/222] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:44.903812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:44.903812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:44.903812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:44.920147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:44.921879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:45.000698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 115/223] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:45.001619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.012001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718ff3e-5f44-4cdd-b37f-46d82b9210aa None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.052491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.053453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.053635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.053903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.055151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.056608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.058381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 109/224] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:44 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:45.059181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.128745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee1610-b218-49f8-8e92-188b9a322958 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.172226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.172807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.173094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.173370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.174137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:45.174474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.176121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.189641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 116/225] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:45.190988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.201677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd8c76-c756-4b3d-8f31-86c55a511023 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.262678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.263496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.263782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.264576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.278885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:45.289826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:45.290813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:45.291606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:45.294297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 110/226] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:45.295549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.326765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fbe7-cdad-4dff-9a81-1742aec83258 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.371580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.372245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.372527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.372802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.373651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=226bc788b84346638e5bcbeb38947aae) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:45.373962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.376006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.382458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 117/227] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/role_assignments?user.id=226bc788b84346638e5bcbeb38947aae&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:45.391263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.447200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073769b4-aad1-40dc-a4bc-647ba3267e09 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.506009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.506956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.507243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.507522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.512705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.514256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.516396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 111/228] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:45.543747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.628855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72d376-f88c-4b64-a60c-533720beeea6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.679742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.680452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.680741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.681017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.681790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:45.682101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.684213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.696864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 118/229] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:45.707061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.721899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d237b-151a-4f2f-a91e-0884686979b7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.767399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.768284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.768572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.768856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.770149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.771750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.773843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 112/230] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:45.779508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:45.885788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7b5c7-abcc-4fe5-bfb3-adf1d010a60a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.896884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800433c-d23b-4ceb-97dc-a0ac21d75ca9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:45.931541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.932452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.932737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.933015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb41f77c34c24422beb85b825bf2ddb5/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.935377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user_id=cb41f77c34c24422beb85b825bf2ddb5, role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.936915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.948418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:45.957538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 119/231] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:45.960854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:46.028354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 113/232] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:45 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb41f77c34c24422beb85b825bf2ddb5/roles/4642f59bea334153a5500859e246a0c6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:56:46.073527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bada32-c563-4f33-8fc8-c2d166b58138 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:46.122167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:46.123512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:46.123998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:46.124831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:46.127763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:46.129316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a477911-8fa6-45ff-8955-53384b93d0cd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:46.131570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 120/233] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:45 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:46.138378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:46.272771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758bc09-ef9c-4a70-aae9-281f65e51b6c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:46.312607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:46.313594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:46.313900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:46.314158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb41f77c34c24422beb85b825bf2ddb5/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:46.317669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user_id=cb41f77c34c24422beb85b825bf2ddb5, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:46.319158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217d06a-4ccc-447c-ab6b-cf2953ffdd75 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:46.429796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 114/234] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:46 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb41f77c34c24422beb85b825bf2ddb5/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:47.486846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a76d3-4930-45ed-a8b4-59587edb03a3 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:47.487557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a76d3-4930-45ed-a8b4-59587edb03a3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:47.487557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a76d3-4930-45ed-a8b4-59587edb03a3 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:47.489077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 121/235] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:47.494370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28a21-91cc-440b-850f-d202b7759363 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:47.495269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28a21-91cc-440b-850f-d202b7759363 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:47.495740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28a21-91cc-440b-850f-d202b7759363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:47.519515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-23e28a21-91cc-440b-850f-d202b7759363 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:47.521164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-23e28a21-91cc-440b-850f-d202b7759363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:47.610902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 115/236] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:47.648431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a76d3-4930-45ed-a8b4-59587edb03a3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:47.755966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c581d-0e95-4c47-bd8b-9f92cab7defb None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:47.756159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c581d-0e95-4c47-bd8b-9f92cab7defb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:47.756438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c581d-0e95-4c47-bd8b-9f92cab7defb None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:47.757891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 116/237] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:47.763624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:47.763911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:47.764187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:47.777946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a76d3-4930-45ed-a8b4-59587edb03a3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:47.806923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:47.808177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:47.847180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:47.847985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:47.848272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:47.848550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:47.859815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:47.867828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:47.868763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:47.870095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 122/238] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:47.881855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:47.980659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 117/239] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:47.996240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed4985-6b92-48ea-8fd4-66d237db5a45 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.021699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.032882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.033566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.033853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.034129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.034987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:48.035298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.036785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.045754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 123/240] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:48.052293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.173008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea02ada-61cb-4422-a294-eb7547202090 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.213744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.215819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.216290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.216715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] PATH_INFO: `/v3/users/cb576530e4f44d8eb92850303d2d5ddc` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.233058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=cb576530e4f44d8eb92850303d2d5ddc)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.234918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.237769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 124/241] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/users/cb576530e4f44d8eb92850303d2d5ddc => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:48.244192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.287704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae1f83-5f84-4202-801e-3be2bec960d9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.346440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.347717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.348141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.348740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.350874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.352826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7554689-7e0d-4bbf-b2b2-105752a4fa62 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.356120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 118/242] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:48.362225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d1fab-9921-44ed-bf34-8fac72f4c902 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.363051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d1fab-9921-44ed-bf34-8fac72f4c902 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.363051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d1fab-9921-44ed-bf34-8fac72f4c902 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.364498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 119/243] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:48.365511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d1fab-9921-44ed-bf34-8fac72f4c902 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.400528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305819de-df54-49fd-8cb1-54487eb4dd8d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.443485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.444520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.445005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.445496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.501511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d1fab-9921-44ed-bf34-8fac72f4c902 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.522329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.524486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac2f3a-202f-42de-ae81-9db8ada75efc None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.545943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.548215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.548504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.548780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.549494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.551680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.635266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 125/244] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:48.636869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.637163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.637442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.659258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:48.660468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:48.669401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 120/245] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:48 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:48.670685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.758384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 126/246] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:48.794057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.844921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6dc3-c862-4e58-af75-de138a54e343 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:48.906665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:48.908541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.909315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.909786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.911461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:48.913965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d989db-5263-4c64-aff5-fd67d621476f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:48.916412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 121/247] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:48.923003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1922-19a9-4df9-a061-bd55faa3cd21 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:48.923624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1922-19a9-4df9-a061-bd55faa3cd21 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:48.924048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1922-19a9-4df9-a061-bd55faa3cd21 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:48.925657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 122/248] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:48.928454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed1922-19a9-4df9-a061-bd55faa3cd21 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:48.967001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd854f-7d85-421f-aebe-98a82b38a7b7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.015710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.017038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.017462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.018029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.030841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-260004cc-cb10-43ea-a220-c92309008dba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:49.040488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:49.041441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:49.042278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 127/249] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:49.043466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.043956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.044399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.065949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:49.067208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:49.074626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed1922-19a9-4df9-a061-bd55faa3cd21 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.131851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.132527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.132813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.133270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.134350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb576530e4f44d8eb92850303d2d5ddc) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:49.134811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:49.136425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:49.147711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 123/250] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:48 2022] GET /identity/v3/role_assignments?user.id=cb576530e4f44d8eb92850303d2d5ddc&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:49.148948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.180297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 128/251] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:49.231754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.278610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bc10-1663-4f4e-a05a-721991255f0b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.326580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.327272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.327551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.327920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.329112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:49.329422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:49.336353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:49.361316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 124/252] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:49.369054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.494053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea245f5-2eb5-4892-b729-6bb97a8def13 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.519689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77612eef-9386-419d-a772-b1bf3dd7ef58 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.559232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.560412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.560867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.561337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] PATH_INFO: `/v3/users/226bc788b84346638e5bcbeb38947aae` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.563811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] RBAC: Authorizing `identity:get_user(user_id=226bc788b84346638e5bcbeb38947aae)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:49.565479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:49.567826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 125/253] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/users/226bc788b84346638e5bcbeb38947aae => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:49.577200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.580202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.581650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.582139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.582669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.600118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:49.608759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:49.610610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:56:49.611407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:49.613982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 129/254] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:49.619737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.790262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9284-b254-4f5f-b428-010d743a0832 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.836678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.837727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.838223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.838695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.839680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:49.840193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:49.841797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94786ee8-c30f-4946-aa39-31fb3ccef239 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:49.860335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 130/255] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:49.865663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0208-92b9-4bfc-9cd3-a46540dec904 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.867719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0208-92b9-4bfc-9cd3-a46540dec904 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.868199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0208-92b9-4bfc-9cd3-a46540dec904 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.870832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 131/256] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:49.873980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b0208-92b9-4bfc-9cd3-a46540dec904 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:49.880579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3f1c1-cd8e-4b2c-b44f-fc1349560805 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:49.943997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:49.944886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.945191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.945614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.948392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:49.949873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e8ce7-8e51-469b-ab97-ca41299989f7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:49.952250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 126/257] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:49.958317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:49.958317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:49.958317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:49.992606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:49.993825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:50.053288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b0208-92b9-4bfc-9cd3-a46540dec904 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.110784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:50.110784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:50.110784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:50.110784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:50.111196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:50.111196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:50.111196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 132/258] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:49 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:50.111726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.157033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 127/259] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:50.158251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.222230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f42b2-32c2-4751-a67a-2061b8ee5254 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.255802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:50.256706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:50.256990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:50.257264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/226bc788b84346638e5bcbeb38947aae/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:50.259377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user_id=226bc788b84346638e5bcbeb38947aae, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:50.260761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:50.307049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f95da-3ef1-42dc-8c47-548e825d3b5c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.346683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:50.347823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:50.348441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:50.349293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:50.360989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 133/260] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:50 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/226bc788b84346638e5bcbeb38947aae/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:50.366229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.555170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:50.556659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:50.568009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae059f7c-30ea-4e71-b08e-f5937fd44b0d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.610652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:50.613325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:50.613885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:50.614597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:50.629698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:50.639209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:50.640288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:56:50.641711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:50.644759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 134/261] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:50.651100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.819458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 128/262] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:50.825585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.862139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d0c5-32c2-4501-9e39-9f05937c0690 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.909902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:50.910614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:50.910895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:50.911172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:50.911931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:50.912236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:50.913636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:50.940819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 135/263] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:50.946699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:50.955795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2fa8a-587d-4d77-8585-c3a28741de8d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:50.999075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.000514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.001051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.001327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.003349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.004839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.006682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 129/264] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:50 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:51.012972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:51.145949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412b167-dc1a-4f34-960a-bb8a054943d3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:51.182000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.183059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.183373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.183622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] PATH_INFO: `/v3/users/cb41f77c34c24422beb85b825bf2ddb5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.189559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] RBAC: Authorizing `identity:get_user(user_id=cb41f77c34c24422beb85b825bf2ddb5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.191092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.197378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 136/265] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:50 2022] GET /identity/v3/users/cb41f77c34c24422beb85b825bf2ddb5 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:51.200895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:51.261986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76927ee-6560-4db2-be55-db0b3c9978c0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:51.321341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.322007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.322337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.322617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.323456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb41f77c34c24422beb85b825bf2ddb5) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:51.323768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.325222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.332770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 130/266] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:51 2022] GET /identity/v3/role_assignments?user.id=cb41f77c34c24422beb85b825bf2ddb5&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=4642f59bea334153a5500859e246a0c6 => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:51.360077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2c95-d8a9-4c2b-a0b0-acd29ba8fde0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:51.399094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.399695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.400162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.400441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.475420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.476957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.593833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 137/267] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 13:56:51.595653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:51.744293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e003f6-b5c2-4f40-b734-dc21c5be67fc None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:51.790675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.791557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.791840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.792104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.793352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.794840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40f0b0-fa8f-47ea-9eb2-6234e076bffb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.797601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 131/268] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:51 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:51.806765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:51.936529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a955035-e92c-45a9-b332-a7de4583e664 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:51.984755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:51.985344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:51.985627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:51.985880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:51.986756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb41f77c34c24422beb85b825bf2ddb5) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:51.987069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:51.988754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b1b06-4c0f-4585-8191-3e7f2b46a7c9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:51.995625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 138/269] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:51 2022] GET /identity/v3/role_assignments?user.id=cb41f77c34c24422beb85b825bf2ddb5&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:52.217037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1855b-d36c-40c3-9c3e-f9c88638fa42 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.217309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1855b-d36c-40c3-9c3e-f9c88638fa42 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.217683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1855b-d36c-40c3-9c3e-f9c88638fa42 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.220062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 132/270] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:52.225471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.225825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.226073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.244916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:52.245604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd685-f7bf-4097-bf27-918957d24316 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.245895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd685-f7bf-4097-bf27-918957d24316 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.246079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:52.246187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd685-f7bf-4097-bf27-918957d24316 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.247735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 133/271] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:56:52.253876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.254166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.254467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.273038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:52.276173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:52.331083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 139/272] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:52.362756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 134/273] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:52.364657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.386043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.471069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de34a55-9856-40dc-8095-969ba8e8eea4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.497172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24173788-f2c0-4fd6-b85a-2967843adfa3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.511858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.512674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.512957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.513232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.524840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:52.532823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:52.533771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:52.534587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 140/274] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:52.540082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.541229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.541335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.541890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.542450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.554993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:52.564353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:52.565280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:52.566058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:52.568869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 135/275] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:52.576572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.656180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f44f9-4ab6-4c75-852e-844377d30877 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.689328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4d5f3-a29a-4bc0-8a03-e2b4fdbfbc32 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.696865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.697658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.698072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.698500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.699978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:52.700349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:52.701747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:52.718312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 141/276] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:52.724556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.745376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.746064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.746393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.746672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.747683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:52.747936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:52.749750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:52.762053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 136/277] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:52.768995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.852414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf3-c528-414b-8094-9435a80b9424 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.894897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aab6f-0c69-42c6-b590-b42a5241b5bf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:52.895606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.896483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.896759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.897028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] PATH_INFO: `/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.913155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] RBAC: Authorizing `identity:get_user(user_id=5eef3f6430d041ea9bb6b8c4027b7af8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:52.915638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69354f24-f9c2-4a64-9eac-a207834cf3cb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:52.923534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 142/278] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:52.923534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280bb5a-14df-43ba-a7e6-1106d10b6274 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.923534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280bb5a-14df-43ba-a7e6-1106d10b6274 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.923534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280bb5a-14df-43ba-a7e6-1106d10b6274 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.923534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 143/279] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:52.925762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280bb5a-14df-43ba-a7e6-1106d10b6274 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:52.940326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:52.941348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.941626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.941901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] PATH_INFO: `/v3/users/cb576530e4f44d8eb92850303d2d5ddc` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.943145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb576530e4f44d8eb92850303d2d5ddc)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:52.944612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2c789-bdfa-4ea8-a282-c3f19d19f7b3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:52.950769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 137/280] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/users/cb576530e4f44d8eb92850303d2d5ddc => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:52.950769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:52.950769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:52.950769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:52.967114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:52.968290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:53.041375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280bb5a-14df-43ba-a7e6-1106d10b6274 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.047376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 138/281] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:53.048805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:53.087197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.088499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.089014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.089642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.091049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, user.id=5eef3f6430d041ea9bb6b8c4027b7af8, scope.system=all) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:53.091838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:53.094033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:53.110574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 144/282] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 10 13:56:52 2022] GET /identity/v3/role_assignments?user.id=5eef3f6430d041ea9bb6b8c4027b7af8&scope.system=all&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:53.110574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:53.204462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554079e0-4c6a-49d0-af0c-3ab9669d6f63 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.233520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6343f6-698d-474a-b940-f77859945cf1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.254614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.255967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.256322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.256667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.258066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:53.260326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:53.263069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 139/283] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:53 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:53.269340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:53.283130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.284052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.284624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.285124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.287352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:53.289242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:53.296428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 145/284] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:53.298326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:53.399667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054ced-b3c0-4652-986d-88f63a223788 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.424725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043efe8-49b9-47ee-abbf-ef10848d4e50 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.438607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.440036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.440451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.440844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb576530e4f44d8eb92850303d2d5ddc/roles/eea647ad81e945d6a7b6f7ca348970bf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.443890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user_id=cb576530e4f44d8eb92850303d2d5ddc, role_id=eea647ad81e945d6a7b6f7ca348970bf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:53.446431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6222a-6af4-4e12-bf39-aabb133e48da None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:53.460342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df324649-7244-41e3-933b-d107a13832bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.461027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324649-7244-41e3-933b-d107a13832bd None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.461311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324649-7244-41e3-933b-d107a13832bd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.461590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324649-7244-41e3-933b-d107a13832bd None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.462348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df324649-7244-41e3-933b-d107a13832bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:53.463950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df324649-7244-41e3-933b-d107a13832bd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:53.527410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 140/285] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:56:53 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/users/cb576530e4f44d8eb92850303d2d5ddc/roles/eea647ad81e945d6a7b6f7ca348970bf => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:56:53.534429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74bba-54a3-44e2-9c2c-e63f6e1fcbf8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.534429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74bba-54a3-44e2-9c2c-e63f6e1fcbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.534429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74bba-54a3-44e2-9c2c-e63f6e1fcbf8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.534429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 141/286] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:53.538645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.538929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.539206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.561893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 146/287] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:56:53 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:53.562074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:53.562074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:53.663285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 142/288] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:56:53.689279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df324649-7244-41e3-933b-d107a13832bd None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:53.825679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df324649-7244-41e3-933b-d107a13832bd None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:53.862164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:53.863022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:53.863310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:53.863575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:53.878468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:53.887941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:53.888917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:53.889694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:53.893138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 147/289] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:53.899468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:54.025925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662ebae-6716-4653-bb66-d816ea84b8f5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:54.066477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:54.067290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:54.067701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:54.067975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:54.068742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:54.069052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:54.070466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:54.081743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 143/290] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:54.087254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:54.194655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec7020-29a4-4121-a8b7-a66a477ca6ec None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:54.229986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:54.230902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:54.231185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:54.231461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] PATH_INFO: `/v3/users/226bc788b84346638e5bcbeb38947aae` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:54.232695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] RBAC: Authorizing `identity:get_user(user_id=226bc788b84346638e5bcbeb38947aae)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:54.234267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:54.236346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 148/291] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:54 2022] GET /identity/v3/users/226bc788b84346638e5bcbeb38947aae => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:54.245136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:54.367442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9364bb-c489-4895-8e3a-d003741d3300 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:54.416897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:54.417567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:54.417854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:54.418135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:54.489866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:54.491385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:54.597139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 144/292] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:54.603056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:54.731243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d60f1-60b9-437a-8433-327ddb00e2ef None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:54.766047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:54.767479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:54.767763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:54.768043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:54.769269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:54.770825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb598e4-5f6b-466f-b4eb-e0cc0c3123f5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:54.772991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 149/293] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:54 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:54.778370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:54.928524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd13eff-78bc-4408-81b6-a61e89ec5a99 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:54.973800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:54.974568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:54.974792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:54.975070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:54.975922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=226bc788b84346638e5bcbeb38947aae) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:54.976388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:54.977829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d884c-f981-4ab8-9666-8625d93ccb96 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:54.985123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 145/294] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:54 2022] GET /identity/v3/role_assignments?user.id=226bc788b84346638e5bcbeb38947aae&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:55.682565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd4ef-320a-4355-be6c-42c87a66a5e4 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:55.683112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd4ef-320a-4355-be6c-42c87a66a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:55.683112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd4ef-320a-4355-be6c-42c87a66a5e4 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:55.684771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 150/295] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:55.690585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:55.690876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:55.691334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:55.719735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:55.719735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:55.799172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 146/296] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:55.821584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fd4ef-320a-4355-be6c-42c87a66a5e4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.090302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fd4ef-320a-4355-be6c-42c87a66a5e4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.150157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.151321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.151789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.152272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.165842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:56.174122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:56.175962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:56.176748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:56:56.179664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 151/297] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:56.185850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.293675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f207634-09cc-4638-9aea-95f4b61a0493 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.330886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.331870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.332337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.332818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.333837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:56.334579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:56.337228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ddba3-d2ae-4a17-95c9-e8b8c6e423bc None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:56.349582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 147/298] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:56.357683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.385103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75beb-8df7-48cc-b02b-4e7c9a7555c9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.385675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75beb-8df7-48cc-b02b-4e7c9a7555c9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.386133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75beb-8df7-48cc-b02b-4e7c9a7555c9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.391931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 148/299] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:56.393439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.393933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.394422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.431962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:56.435259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:56.498781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b07cd-94eb-4a6d-a363-b6af501a77cd None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.527943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 149/300] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:56.543632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.544968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.545475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.545996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] PATH_INFO: `/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.547753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] RBAC: Authorizing `identity:get_user(user_id=5eef3f6430d041ea9bb6b8c4027b7af8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:56.549441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:56.551683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.553477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 152/301] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:56 2022] GET /identity/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:56.559970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.668727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f5a3d-52b9-4f4e-9e18-0a96b0c1aa09 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.701104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94e19-86ec-40af-ac0a-2c03521fb0ab None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.710044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fa901-8204-4955-9932-543640437612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.710882 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fa901-8204-4955-9932-543640437612 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.711160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fa901-8204-4955-9932-543640437612 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.711433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fa901-8204-4955-9932-543640437612 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.723025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fa901-8204-4955-9932-543640437612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-cb2fa901-8204-4955-9932-543640437612 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:56.730781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:56.731778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:56.732562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 150/302] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:56:56.733675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70547c-6e87-4a9a-9d33-439a90361d9b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.733675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70547c-6e87-4a9a-9d33-439a90361d9b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.733791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70547c-6e87-4a9a-9d33-439a90361d9b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.735811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 151/303] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:56.736732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05edd13-ca07-4ddd-97f0-8ceab43d3587 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.737158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05edd13-ca07-4ddd-97f0-8ceab43d3587 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.737832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05edd13-ca07-4ddd-97f0-8ceab43d3587 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.739247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 152/304] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:56.741017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05edd13-ca07-4ddd-97f0-8ceab43d3587 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.741907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.742821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.743192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.743475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] PATH_INFO: `/v3/system/users/5eef3f6430d041ea9bb6b8c4027b7af8/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.745002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5eef3f6430d041ea9bb6b8c4027b7af8, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:56.746439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6b8c5-df94-481f-8998-c1294b423b80 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:56.818903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 153/305] 10.222.0.58 () {58 vars in 1334 bytes} [Tue May 10 13:56:56 2022] PUT /identity/v3/system/users/5eef3f6430d041ea9bb6b8c4027b7af8/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:56:56.821359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.821729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.822069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.840217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:56.841448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:56.846372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05edd13-ca07-4ddd-97f0-8ceab43d3587 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:56.887371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:56.888072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.888355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.888633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.889416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:56.889724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:56.891489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f252e09-8f0f-4052-a8cf-bfc5180fe009 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:56.905487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 153/306] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:56.905487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:56.905487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:56.905487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:56.924545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 154/307] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:56.925686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:56.935759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:56.937224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:57.013297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 154/308] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:57.014221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.043349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80345d-b6d1-42df-8c9f-2c5712002d6b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.080007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.080882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.081166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.081441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] PATH_INFO: `/v3/users/cb576530e4f44d8eb92850303d2d5ddc` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.082665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] RBAC: Authorizing `identity:get_user(user_id=cb576530e4f44d8eb92850303d2d5ddc)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.084130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.090653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 155/309] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:56 2022] GET /identity/v3/users/cb576530e4f44d8eb92850303d2d5ddc => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.090653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.129794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e143246-fd00-4c16-8967-b60da4875ddc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.172960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.173775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.174061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.174371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.192329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:57.199540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:57.200456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:56:57.201439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:57.202052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4a96-c1ed-4bdc-87b1-f23bb337c419 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.202124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 155/310] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:57.202124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.240854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.242048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.242547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.242996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.244488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.246796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.254732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 156/311] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.254732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.322950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60885d-e76c-456a-a632-110e965f8092 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.361271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2012-dc35-4916-ab2a-37398ceb2281 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.363134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.363810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.364091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.364368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.397732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.398731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.399018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.399299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.400055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:57.400366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.401774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.416171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 157/312] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.416171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.445157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.446983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.524204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939b36-8308-4c80-89de-fae2886fba5a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.559162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.559836 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.560122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.560399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] PATH_INFO: `/v3/groups` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.560979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 156/313] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:56:57.561200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.561827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.562706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.641214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 158/314] 10.222.0.58 () {60 vars in 1134 bytes} [Tue May 10 13:56:57 2022] POST /identity/v3/groups => generated 229 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:56:57.643013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcd9c-1722-4faf-b7d0-fb872b6bea7f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.708214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.709498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.710067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.710547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] PATH_INFO: `/v3/users/1f3310ccf60f41a29a370920293b9e68` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.728174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] RBAC: Authorizing `identity:get_user(user_id=1f3310ccf60f41a29a370920293b9e68)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.729938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.732152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 157/315] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/users/1f3310ccf60f41a29a370920293b9e68 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.742370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.766249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871f3f1-e239-4aa0-b6ae-138c51fd8e23 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.798089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.799023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.799308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.799589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.800947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.802496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.804408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 159/316] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.810499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:57.859885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381dc8-9091-45d8-9712-3af41e1ac89a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.903833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.904696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.904988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.905269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.906138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, user.id=1f3310ccf60f41a29a370920293b9e68, scope.system=all) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:57.906568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.908094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76569ca5-9e52-424c-8693-d21605fb7cd3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:57.914060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 158/317] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/role_assignments?user.id=1f3310ccf60f41a29a370920293b9e68&scope.system=all&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:57.940727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aee68-06d9-4880-94df-a3a5c1e75499 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:57.982138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:57.982804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:57.983093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:57.983374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:57.984803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, user.id=cb576530e4f44d8eb92850303d2d5ddc) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:57.985180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:57.986765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9775d59-f412-4699-854f-260a94059dbb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:58.006282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 160/318] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:56:57 2022] GET /identity/v3/role_assignments?user.id=cb576530e4f44d8eb92850303d2d5ddc&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:59.174618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901667-33bf-4677-a752-46feb85c6059 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.175169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901667-33bf-4677-a752-46feb85c6059 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.175324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901667-33bf-4677-a752-46feb85c6059 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.176858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 159/319] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:56:59.182336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.182832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.182910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.199821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:56:59.201019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:56:59.288832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 161/320] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:56:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:56:59.311762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901667-33bf-4677-a752-46feb85c6059 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:59.414058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901667-33bf-4677-a752-46feb85c6059 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:59.448077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:59.448797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.449074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.449343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.460706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:56:59.468559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:56:59.475555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:56:59.476399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 160/321] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:56:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:56:59.478661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:59.591415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a99f3c-7da6-474e-a12d-d31833c3256a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:59.626816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:59.627489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.627771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.628042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.628794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:59.629099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:59.631180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:59.645200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 162/322] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:56:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:56:59.646415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:59.765012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dd2e9-9b77-4ae7-b797-5c360cf94b24 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:59.801795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:59.802714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.802996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.803270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] PATH_INFO: `/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.804508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] RBAC: Authorizing `identity:get_user(user_id=5eef3f6430d041ea9bb6b8c4027b7af8)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:59.805943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8678a-a1c4-4564-b4e3-8eb5952c1e8a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:59.807809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 161/323] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:56:59 2022] GET /identity/v3/users/5eef3f6430d041ea9bb6b8c4027b7af8 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:56:59.814275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:56:59.929391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cf982-7b10-4387-af4f-387819e90d4e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:56:59.970421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:56:59.971084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:56:59.971413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:56:59.971645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:56:59.972490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] RBAC: Adding query filter params (role.id=88a0e9d0fe9d48eaae419a9d79677539, user.id=5eef3f6430d041ea9bb6b8c4027b7af8, scope.system=all) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:56:59.972800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:56:59.974259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a994587-4050-44d5-a6ea-67be1d16f35d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:56:59.994235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 163/324] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 10 13:56:59 2022] GET /identity/v3/role_assignments?user.id=5eef3f6430d041ea9bb6b8c4027b7af8&scope.system=all&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:00.044694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f73f1-e275-4f38-bcd7-ed60dcfda189 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.044980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f73f1-e275-4f38-bcd7-ed60dcfda189 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.045256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f73f1-e275-4f38-bcd7-ed60dcfda189 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.046975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 162/325] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:00.052782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ef32f-b650-47e8-bace-9e46a4d9cd7f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.053124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ef32f-b650-47e8-bace-9e46a4d9cd7f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.053351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ef32f-b650-47e8-bace-9e46a4d9cd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.071581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-933ef32f-b650-47e8-bace-9e46a4d9cd7f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:00.072813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-933ef32f-b650-47e8-bace-9e46a4d9cd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:00.151196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 164/326] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:00.174131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f73f1-e275-4f38-bcd7-ed60dcfda189 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.209935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca56051-5a23-47a6-96df-ef9b6678708a None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.210475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca56051-5a23-47a6-96df-ef9b6678708a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.210574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca56051-5a23-47a6-96df-ef9b6678708a None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.211913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 165/327] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:57:00.217396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.217680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.217954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.238022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:00.239177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:00.292571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f73f1-e275-4f38-bcd7-ed60dcfda189 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.310905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 166/328] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:00.328841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.329648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.329930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.330227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.331447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:00.333095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.333249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:00.334829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 163/329] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:00.340244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.434016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875ca64-ea9c-45a7-8067-2128195e2764 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.442407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fcf9e-bd8c-46b9-ac24-4c7dae4d559b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.472429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.473226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.473508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.473778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.476509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.477177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.477460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.477769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] PATH_INFO: `/v3/groups` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.478483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:00.479945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:00.483963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:00.492285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:00.493246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:00.494038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 167/330] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:00.499202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.584307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 164/331] 10.222.0.58 () {60 vars in 1134 bytes} [Tue May 10 13:57:00 2022] POST /identity/v3/groups => generated 226 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:57:00.599366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b17b8-97ea-4bd6-830a-f14151dbcdc3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.638131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.638845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.639131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.639409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.640187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:00.640503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:00.641918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:00.656048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 168/332] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:00.658089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.768455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8135e3-9238-47c7-a761-3601bab36a8f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.803271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.804151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.804435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.804707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] PATH_INFO: `/v3/users/1f3310ccf60f41a29a370920293b9e68` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.805924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] RBAC: Authorizing `identity:get_user(user_id=1f3310ccf60f41a29a370920293b9e68)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:00.807525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6654a-4115-4471-ad30-bbfad78bc6dd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:00.809227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 165/333] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:57:00 2022] GET /identity/v3/users/1f3310ccf60f41a29a370920293b9e68 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:00.815340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:00.934035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6aabc-86a3-48f1-83d6-5f31a106f288 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:00.985092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:00.986487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:00.986910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:00.987317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] PATH_INFO: `/v3/system/users/1f3310ccf60f41a29a370920293b9e68/roles/eea647ad81e945d6a7b6f7ca348970bf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:00.989676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f3310ccf60f41a29a370920293b9e68, role_id=eea647ad81e945d6a7b6f7ca348970bf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:00.991785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f217ea-a625-45a2-91af-68db6821f736 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:01.088619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 169/334] 10.222.0.58 () {58 vars in 1334 bytes} [Tue May 10 13:57:00 2022] PUT /identity/v3/system/users/1f3310ccf60f41a29a370920293b9e68/roles/eea647ad81e945d6a7b6f7ca348970bf => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:57:03.171046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58695-b5cb-4d73-90f4-8a7ad8537fc0 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.171594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58695-b5cb-4d73-90f4-8a7ad8537fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.171594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58695-b5cb-4d73-90f4-8a7ad8537fc0 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.174393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 166/335] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:03.181068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.182028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.182553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.207434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:03.208990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:03.306003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 170/336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:03.345313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d58695-b5cb-4d73-90f4-8a7ad8537fc0 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:03.479061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d58695-b5cb-4d73-90f4-8a7ad8537fc0 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:03.518611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:03.519428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.519708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.519982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.536396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d3354423-a504-4b74-8386-67be07718bd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:03.544226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:03.545151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:03.545985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:03.548707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 167/337] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:03.555677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:03.647660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927524-c4b2-4805-944d-c03bd4431afc None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.647942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927524-c4b2-4805-944d-c03bd4431afc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.648210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927524-c4b2-4805-944d-c03bd4431afc None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.649963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 168/338] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:03.667725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.668349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.668850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.695846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:03.697431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:03.715324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73dd64-302c-4bca-9276-eaad2ff41bc8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:03.773537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:03.774224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.774666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.775088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:03.776066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:03.776533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:03.778063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:03.788405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 171/339] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:03.794176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:03.833427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 169/340] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:03.883706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:03.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bfac3-bfd6-4dd2-bfb2-b900b3f367f9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:03.967053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:03.968041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:03.968494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:03.968919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.025910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7dea6-1fd9-4aa4-88d3-8bbd8e174226 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.073511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.075361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.076269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.076622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.076976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.077670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.089611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:04.102260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:04.103430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:04.104220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:04.105843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 170/341] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:04.110996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.198579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 172/342] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 13:57:04.201409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.218096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bc7f-acae-4cd0-ae3e-e2a69de2885c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.261761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.262478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.262760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.263196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.264125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:04.264591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.266734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.281065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 171/343] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:04.287525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.309688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93979-ed7f-4a99-a9b9-7561aefedcac None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.355686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.356567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.356847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.357122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.363083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.364587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cad506-3cc7-44e3-8ed0-1ddbd2d66694 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.366840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 173/344] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:04.368167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ece2ad-dd73-40e2-80da-698fb33a8e99 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.368458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ece2ad-dd73-40e2-80da-698fb33a8e99 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.368737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ece2ad-dd73-40e2-80da-698fb33a8e99 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.370320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 174/345] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:04.376432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ece2ad-dd73-40e2-80da-698fb33a8e99 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.418822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0028-daf8-4a99-9be8-ff1b4a8eb7a8 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.466972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.467655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.468053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.468241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.519992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ece2ad-dd73-40e2-80da-698fb33a8e99 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.552880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.554296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4ac51-f585-4293-b350-3f95c66281e0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.570917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.571796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.572079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.572354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.579342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.580803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f875-ac13-44e7-850e-1ab154b87b15 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.583144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 175/346] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:04.584393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab962-8a7e-4ce4-82b6-0fa0ad3c8a2a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.584679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab962-8a7e-4ce4-82b6-0fa0ad3c8a2a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.584956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beab962-8a7e-4ce4-82b6-0fa0ad3c8a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.607208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4beab962-8a7e-4ce4-82b6-0fa0ad3c8a2a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:04.608428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4beab962-8a7e-4ce4-82b6-0fa0ad3c8a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 172/347] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6826-9661-4dbc-b214-93526d755a1e None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6826-9661-4dbc-b214-93526d755a1e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6826-9661-4dbc-b214-93526d755a1e None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 173/348] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:04.674457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca6826-9661-4dbc-b214-93526d755a1e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.699818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 176/349] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:04.700459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.700746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.701020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.739574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:04.741156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:04.830903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 177/350] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:04.831726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.839671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca6826-9661-4dbc-b214-93526d755a1e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.894504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.895167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.895445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.895716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.896545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=381a803ca3e2428db7752680e6ae1d58) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:04.896851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.898378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:04.906072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 174/351] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:04.907750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:04.951163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f5ce5-cc59-4bf1-b175-e8e15ad9350d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:04.994563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:04.995466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:04.995752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:04.996033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:04.997565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:04.999087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.001614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 178/352] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.002727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.118398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c0cb6-3b75-458a-bf79-e6041873ebf3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.158470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e5e3-9760-4b0b-bc6d-3a7c0893c2d9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.171488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.173727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.174166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.174610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.189544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:05.199965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:05.201132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:05.202131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:05.204251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.205385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.205937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.206455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.209045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 175/353] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:05.210142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.221014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:05.229894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:05.230958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 10 13:57:05.231745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:05.234268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 179/354] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:05.235408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.346441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2be26-9d80-4a88-b956-dd8f30ffc89e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.361670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c05615-5a57-4d70-bab1-d5e1c743878f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.403493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.404747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.405236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.405738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.408023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.411080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.414419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 176/355] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.418228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.418228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.418228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.418228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.418586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:05.418586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.419576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.419682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.434296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 180/356] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.435631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.549488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2787ad-3430-42e5-bf00-f92f7dbd2306 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.604319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad931-a783-4528-a690-d061c6176b15 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.616811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.617503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.617790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.618068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.618868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:05.619216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.620662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.636591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 177/357] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.639363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.692238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.693050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.693334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.693607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.694477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=381a803ca3e2428db7752680e6ae1d58) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:05.694786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.696347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.702659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 181/358] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=4642f59bea334153a5500859e246a0c6 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.703558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.762577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f24229-53c6-4993-b2a2-9c78544979fc None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.816024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.816714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.816997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.817275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.824262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dfa43-3652-48c1-b037-fbc2771e32b7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:05.871102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:05.872023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.872307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.872582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] PATH_INFO: `/v3/users/1f3310ccf60f41a29a370920293b9e68` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.874061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=1f3310ccf60f41a29a370920293b9e68)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.876137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dac225-0acb-4cc8-8317-9c1bbc4d0ab3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:05.882637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 182/359] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/users/1f3310ccf60f41a29a370920293b9e68 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:05.884169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd07ce-4126-47e6-9ec6-032192968e5f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:05.884460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd07ce-4126-47e6-9ec6-032192968e5f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:05.884736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd07ce-4126-47e6-9ec6-032192968e5f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:05.886261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 183/360] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:05.892648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd07ce-4126-47e6-9ec6-032192968e5f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:05.912216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:05.913716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da9f2c-a759-4794-a479-83cee08930f6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:06.037595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 178/361] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:06.039097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809cb1-acaf-475c-b12c-cc1de630b72e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.039393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809cb1-acaf-475c-b12c-cc1de630b72e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.039678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809cb1-acaf-475c-b12c-cc1de630b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.059375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4e809cb1-acaf-475c-b12c-cc1de630b72e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:06.061900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4e809cb1-acaf-475c-b12c-cc1de630b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:06.118540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd07ce-4126-47e6-9ec6-032192968e5f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.192301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.192985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.193267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.193542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.194423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] RBAC: Adding query filter params (role.id=eea647ad81e945d6a7b6f7ca348970bf, user.id=1f3310ccf60f41a29a370920293b9e68, scope.system=all) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:06.194743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:06.196228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:06.222074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 184/362] 10.222.0.58 () {56 vars in 1308 bytes} [Tue May 10 13:57:05 2022] GET /identity/v3/role_assignments?user.id=1f3310ccf60f41a29a370920293b9e68&scope.system=all&role.id=eea647ad81e945d6a7b6f7ca348970bf => generated 510 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:06.224547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.229292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 179/363] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:06.245633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99844b-1fe0-4017-8cb5-e8390135cfd2 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.246340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99844b-1fe0-4017-8cb5-e8390135cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.246860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99844b-1fe0-4017-8cb5-e8390135cfd2 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.248595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 180/364] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:57:06.254694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.255305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.255844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.285612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:06.290117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:06.398599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8baace-915d-4c22-8843-cb9138fb6c78 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.441524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.442828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.443340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.444342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.445936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:06.448152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:06.450764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 185/365] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:06.451841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.484086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 181/366] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:06.485319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfd5c4-e9e6-442a-b851-fa0cd984d095 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.635857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.637429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.637980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.638666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.644051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f3d1f-f578-428e-bf81-b393577a7ed2 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.655025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:06.668220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:06.669171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:06.669968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:06.671663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 186/367] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:06.673091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.697233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.697789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.698070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.698389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.699583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:06.701153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:06.702782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 182/368] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:06.704121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.828061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76297b-8e60-4f80-9e1e-ebad34c7d774 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.853377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908602e2-db0b-41d7-8ed1-efca62c9c091 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:06.873700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.875106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.877041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.877041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.890812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:06.896552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:06.897336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:06.897689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:06.898036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:06.898922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:06.899331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:06.900791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:06.902085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:06.903002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:06.903775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:06.906758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 187/369] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:06.908147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:06.914155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 183/370] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:06.914712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.038022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754373-af48-46bc-8903-40d09d27afd9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.039075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514edc68-7b46-4009-b5df-d01aba31edb3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.075949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.076779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.077166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.077541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.078616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:07.079577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.081199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.083284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.084487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.084945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.085358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.086361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:07.086827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.089119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.092957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 184/371] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:07.094014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.097763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 188/372] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:06 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:07.100636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.235856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3b93-88c5-4c22-8b5a-ed3e31df3495 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.278445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.279512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.280024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.280493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.349512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d38b-aab8-4925-aada-637f3af7d3a3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.369311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.372046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.439128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.439785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.440087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.440388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.505058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 189/373] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:07.511867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.565060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.570596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.698259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a372-b979-4297-a42e-2dbae8f18a1d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.746399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.747768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.748261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.748971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] PATH_INFO: `/v3/projects/fab88b77b7fd44528a3ca6841d9f273b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.750841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] RBAC: Authorizing `identity:get_project(project_id=fab88b77b7fd44528a3ca6841d9f273b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.753118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.756106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 190/374] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/projects/fab88b77b7fd44528a3ca6841d9f273b => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:07.771415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.774009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 185/375] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:07.779176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.902856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae7d8c-9349-45eb-a3a5-32b07ea4e5e4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:07.943461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:07.944338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:07.944688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:07.945069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:07.951148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:07.952616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:07.957019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 186/376] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:07.960034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:07.998730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d98c-f64d-4e02-ade6-c08bd08f6197 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.047086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660b235-d467-472a-83e6-f9927806382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.047934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b235-d467-472a-83e6-f9927806382a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.048219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b235-d467-472a-83e6-f9927806382a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.048497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b235-d467-472a-83e6-f9927806382a None admin] PATH_INFO: `/v3/groups/b00947654146476080a7f1c2141af0a2` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.053928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660b235-d467-472a-83e6-f9927806382a None admin] RBAC: Authorizing `identity:get_group(group_id=b00947654146476080a7f1c2141af0a2)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:08.055443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660b235-d467-472a-83e6-f9927806382a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:08.057624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 191/377] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/groups/b00947654146476080a7f1c2141af0a2 => generated 229 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:08.064023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660b235-d467-472a-83e6-f9927806382a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:08.127610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b986fd-2123-436a-a6ea-0cd3bb158925 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.175122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.176320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.176601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.176876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.178121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:08.180456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b7dd9-850d-4b22-801f-cba0c4d5df30 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:08.183803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 187/378] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:07 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:08.185139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f537-eec2-4082-826b-37b4f79962f3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.185423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f537-eec2-4082-826b-37b4f79962f3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.185696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f537-eec2-4082-826b-37b4f79962f3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.187352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 188/379] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:08.193439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f537-eec2-4082-826b-37b4f79962f3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:08.233954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660b235-d467-472a-83e6-f9927806382a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.285263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.285929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.286224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.286501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.289180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] RBAC: Adding query filter params (group.id=b00947654146476080a7f1c2141af0a2, role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=fab88b77b7fd44528a3ca6841d9f273b) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:08.289180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:08.289497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0118c-3916-44ce-ab1e-3d1e43f7a336 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:08.298438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 192/380] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:08 2022] GET /identity/v3/role_assignments?group.id=b00947654146476080a7f1c2141af0a2&scope.project.id=fab88b77b7fd44528a3ca6841d9f273b&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:08.306374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.306374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.306374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.334463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:08.335943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:08.398984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f537-eec2-4082-826b-37b4f79962f3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.483396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.486455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.488122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 193/381] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:08.488661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.489104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.490141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:08.490659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:08.492315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:08.499351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 189/382] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:08 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=4642f59bea334153a5500859e246a0c6 => generated 260 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:08.520511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:08.645758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc375ffd-18ad-40fb-b762-0240fcbd5e41 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.696903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.697975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.698463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.698876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.723626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-9630b0d7-957d-43eb-8b73-83b5141027ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:08.732355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:08.733482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:08.734530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:08.737993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 194/383] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:08.743837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:08.816860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b0ddf-0337-48ad-9fd4-917b374faa31 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.817410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b0ddf-0337-48ad-9fd4-917b374faa31 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.817838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b0ddf-0337-48ad-9fd4-917b374faa31 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.820445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 195/384] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:57:08.826309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.826902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.827386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.851858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:08.853393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:08.869601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4f609-fc50-43ef-a763-c404d021f14a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:08.913772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:08.914855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:08.915358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:08.915831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:08.916785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:08.917295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:08.918983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:08.929588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 190/385] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:08.938874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:08.962104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 196/386] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:09.028791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.110830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4556-4e58-4d50-ab00-0ce0a097e0d1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.191199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde155cd-f805-4cd1-a576-58c50cd1afc8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.196331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.197210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.197492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.197768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.203435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.205000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.207134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 191/387] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:08 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:09.212241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.237033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.238357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.238849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.239350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.260950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:09.271777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:09.273497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:09.274332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 197/388] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:09.283719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.395817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d914bf4-7690-4261-8308-832ecda6cab2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.416663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7dae-a336-4a62-9ce5-79b8bf9825e0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.433995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.435061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.435569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.436065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.437123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:09.437700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.439440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.449702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 198/389] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:09.455946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.495279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.495884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.496169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.496446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.497761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.499233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.505869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 192/390] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:09 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:09.507089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.594561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715ae7-1bde-45bb-bdc3-7ede4f58d655 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.634161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.635827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.636325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.636802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.638492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.640192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.642733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 199/391] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:09 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:09.648150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.775596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414466-ced1-4e60-9434-a199cd99b913 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.809301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58750ae-a93a-43d4-8cf7-f5d9919cb96a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:09.814929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.816799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.817194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.817389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.818745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.820226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 200/392] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:09 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:09.831107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:09.904087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:09.905059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:09.905176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:09.905385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/groups/f4d11e65ad8449abafddae3343fabebf/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:09.911657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, group_id=f4d11e65ad8449abafddae3343fabebf, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:09.913124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60722a-ce12-4e81-851e-14f36873d8cc None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:09.957933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30171bde-1f4c-49b9-b96b-c7fef795a92a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:10.006314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 193/393] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 10 13:57:09 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/groups/f4d11e65ad8449abafddae3343fabebf/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:57:10.008713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:10.009808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.010255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.010791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58/groups/f4d11e65ad8449abafddae3343fabebf/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.013356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a803ca3e2428db7752680e6ae1d58, group_id=f4d11e65ad8449abafddae3343fabebf, role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:10.015073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:10.087640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5da2c0-9e98-4a87-9fe7-d5b835719b71 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.087923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5da2c0-9e98-4a87-9fe7-d5b835719b71 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.088214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5da2c0-9e98-4a87-9fe7-d5b835719b71 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.089809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 194/394] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:57:10.096229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 201/395] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 10 13:57:09 2022] PUT /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58/groups/f4d11e65ad8449abafddae3343fabebf/roles/4642f59bea334153a5500859e246a0c6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:10.097107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.097612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.101594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.127189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:10.127189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:10.234237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 195/396] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:10.259708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:10.418528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcc454-9e01-4e6b-a83f-ab40a0b8b732 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:10.466543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:10.467367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.467649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.467925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.484120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:10.493338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:10.494272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:10.495073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:10.497871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 202/397] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:10.504531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:10.617337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bd0b6-de42-4f4a-a7b2-06f130ac15af None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:10.654054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:10.655183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.655826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.656353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.658032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:10.658587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:10.660197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:10.670376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 196/398] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:10.676810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:10.798138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34964fd-0abf-499a-a351-f3eefeee7ec9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:10.849074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:10.850354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:10.850841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:10.851332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:10.852780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:10.854514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:10.856730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 203/399] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:10 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:10.862424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:10.976925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d156e-0db6-4766-95b0-54e463aa95d0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:11.013302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:11.014591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.015082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.015572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.017483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:11.019174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61f117-6d0c-4e4e-9ac4-b633ccd04cb8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:11.021480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 197/400] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:10 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:11.027165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:11.147092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3aa555-3097-469c-a682-ba554693c8ff None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:11.192430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:11.193661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.194143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.194641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/groups/f4d11e65ad8449abafddae3343fabebf/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.196953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, group_id=f4d11e65ad8449abafddae3343fabebf, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:11.198649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f0b77-ad38-495a-85fd-734cac3e92bf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:11.334963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 204/401] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 10 13:57:11 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/groups/f4d11e65ad8449abafddae3343fabebf/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:11.527484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82d41e-3afb-42bd-bf34-ed63bfd5c967 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.528128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82d41e-3afb-42bd-bf34-ed63bfd5c967 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.528646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82d41e-3afb-42bd-bf34-ed63bfd5c967 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.530705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 198/402] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:11.537113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ed8c-05b0-468c-9590-9a8ef98fa283 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.538077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ed8c-05b0-468c-9590-9a8ef98fa283 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.538586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ed8c-05b0-468c-9590-9a8ef98fa283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.559305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5383ed8c-05b0-468c-9590-9a8ef98fa283 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:11.560853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5383ed8c-05b0-468c-9590-9a8ef98fa283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:11.646497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 205/403] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:11.652545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec35d-f871-4818-b382-b4794d7fae0f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.653085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec35d-f871-4818-b382-b4794d7fae0f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.653533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec35d-f871-4818-b382-b4794d7fae0f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.658842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 199/404] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:11.661573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779773-855d-4385-ad33-2cc435215452 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.661866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779773-855d-4385-ad33-2cc435215452 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.662189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779773-855d-4385-ad33-2cc435215452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.694958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffec35d-f871-4818-b382-b4794d7fae0f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:11.696564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a2779773-855d-4385-ad33-2cc435215452 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:11.697775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a2779773-855d-4385-ad33-2cc435215452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:11.781288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 206/405] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:11.805744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2779773-855d-4385-ad33-2cc435215452 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:11.911330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2779773-855d-4385-ad33-2cc435215452 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:11.912335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffec35d-f871-4818-b382-b4794d7fae0f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:11.952423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:11.953234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.953515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.954328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.956170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:11.956980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:11.957256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:11.957525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:11.978060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:11.982867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:11.986927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:11.987813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:11.988591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 207/406] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:11.991999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:11.994053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:11.996217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 200/407] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:11.997763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.000006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.109101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92be1ab-8c93-4031-822a-3238f9ac41f9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.169535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.170139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.170503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.170780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.171549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:12.171854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.173872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.201326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 208/408] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.216588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.264444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9b08-0ad0-48ee-9f67-6875a2bb2027 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.332416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.333101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.333382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.333657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.334439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:12.334746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.336409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.346955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 201/409] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.352905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.428493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d1c9f-34d1-4387-9902-9f8f18597e20 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.466886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac52a5-4b05-4297-9e11-6d553152c842 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.510060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.510878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.511160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.511492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] PATH_INFO: `/v3/groups/b00947654146476080a7f1c2141af0a2` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.513481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] RBAC: Authorizing `identity:get_group(group_id=b00947654146476080a7f1c2141af0a2)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.514943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.526227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.526227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.526711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.526711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.528547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.529688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.536223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 202/410] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:12 2022] GET /identity/v3/groups/b00947654146476080a7f1c2141af0a2 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.537249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.540332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 209/411] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:12 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.541229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.663875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312a6c9-8a59-4b66-a165-90a24db75abf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.708521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.709319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.709636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.709911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.712768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.714262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.716306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 203/412] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:12 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.726786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.751232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f814e2-bdf9-4362-a9ea-04bf569831c6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.830513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.831388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.831668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.831944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] PATH_INFO: `/v3/projects/fab88b77b7fd44528a3ca6841d9f273b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:12.839135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] RBAC: Authorizing `identity:get_project(project_id=fab88b77b7fd44528a3ca6841d9f273b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:12.840634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:12.842473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 210/413] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:12 2022] GET /identity/v3/projects/fab88b77b7fd44528a3ca6841d9f273b => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:12.851072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:12.936653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cbad-6030-440a-91f1-7e2b4226e3c6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.982314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd5aa-9ba0-41fd-b315-7105a7d7dd47 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:12.991943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:12.993216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:12.993706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:12.994184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/groups/f4d11e65ad8449abafddae3343fabebf/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.000190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd, group_id=f4d11e65ad8449abafddae3343fabebf, role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:13.002659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140ff6e-d115-4b27-9965-cfe8c88d9219 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:13.014242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:13.014242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.014242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.014242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] PATH_INFO: `/v3/projects/fab88b77b7fd44528a3ca6841d9f273b/groups/b00947654146476080a7f1c2141af0a2/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.015435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] RBAC: Authorizing `identity:create_grant(project_id=fab88b77b7fd44528a3ca6841d9f273b, group_id=b00947654146476080a7f1c2141af0a2, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:13.017857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af563af0-029f-4e34-8c55-01a1a6f12a96 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:13.062185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 204/414] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 10 13:57:12 2022] PUT /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd/groups/f4d11e65ad8449abafddae3343fabebf/roles/4642f59bea334153a5500859e246a0c6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:13.091910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 211/415] 10.222.0.58 () {58 vars in 1442 bytes} [Tue May 10 13:57:12 2022] PUT /identity/v3/projects/fab88b77b7fd44528a3ca6841d9f273b/groups/b00947654146476080a7f1c2141af0a2/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:13.252636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e483f2-1633-44fa-897b-5eb1aa976f56 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.253580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e483f2-1633-44fa-897b-5eb1aa976f56 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.254079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e483f2-1633-44fa-897b-5eb1aa976f56 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.255991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 205/416] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:13.261904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc02e9f-2264-4909-b21f-bba51fe61851 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.262223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc02e9f-2264-4909-b21f-bba51fe61851 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.262503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc02e9f-2264-4909-b21f-bba51fe61851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.280396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7bc02e9f-2264-4909-b21f-bba51fe61851 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:13.281622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7bc02e9f-2264-4909-b21f-bba51fe61851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:13.371404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 212/417] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:13.397160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e483f2-1633-44fa-897b-5eb1aa976f56 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:13.513437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eda6-40ae-431e-9d98-43098c682b1b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.513437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eda6-40ae-431e-9d98-43098c682b1b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.513681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23eda6-40ae-431e-9d98-43098c682b1b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.515427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 213/418] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:13.529783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.530111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.530397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.552504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:13.554361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:13.569840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e483f2-1633-44fa-897b-5eb1aa976f56 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:13.635525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:13.636348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.636631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.636912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.655281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:13.665262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:13.666147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:13.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:13.668066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 206/419] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:13.677637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:13.694809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 214/420] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:13.744700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:13.841955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6cff0-1c62-4fdc-a5fd-e826ba457d74 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:13.881253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:13.881914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.882217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.882511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.883796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:13.884115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:13.885660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:13.894961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 207/421] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:13.903295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:13.927941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56b9c6-91ed-475d-a996-5d9da3dd95eb None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:13.967973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:13.968807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:13.969091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:13.969367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:13.993958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:14.001938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:14.003307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:14.005005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:14.011384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 215/422] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:14.016716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.096882 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fdeb2-6c67-415c-ac31-bd1b5ddf553c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.153123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.153944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.154260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.154535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.246630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aacf7-f04e-4b0a-8bf9-8fc51e2a916e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.256860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.258351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98f327-9f3a-4a55-bfe1-9af5e535b3fe None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.286639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.287304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.287586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.287860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.288612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:14.288922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.290317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b24002-63af-4b0d-ae2b-db40caf439da None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.300527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 216/423] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:14.301893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af8568-835e-41ea-89bb-4c678630d6d5 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.302286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af8568-835e-41ea-89bb-4c678630d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.302632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af8568-835e-41ea-89bb-4c678630d6d5 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.308040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 217/424] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:14.308897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af8568-835e-41ea-89bb-4c678630d6d5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.362790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 208/425] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:14.364182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.364569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.364946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.382991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:14.384175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:14.426896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af8568-835e-41ea-89bb-4c678630d6d5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.457160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 209/426] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:14.458121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.475815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.476481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.476764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.477038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.565792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338243f-bb5c-4e1a-bd44-b920fd5f69c7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.569935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.571454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.602903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.603818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.604072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.604354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.605550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.607011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.609053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 210/427] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:14.609830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.682152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 218/428] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:14.682889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.713445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d8a39-2446-48cf-908f-7c9b3c53ef3d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.748385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.749226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.749513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.749832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.762047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:14.770190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:14.771104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 211/429] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:14.772485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.790667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b69fcc-ec47-432b-b644-fc46663eeca9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.827319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.828801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.829303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.829597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.831083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.832555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.839319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 219/430] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:14.840273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.896209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbe080-d1a6-4cdc-a3c7-be6576fff250 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:14.937948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:14.938967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:14.939251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:14.939530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] PATH_INFO: `/v3/projects/381a803ca3e2428db7752680e6ae1d58` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:14.940788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=381a803ca3e2428db7752680e6ae1d58)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:14.943357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:14.945181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 212/431] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/projects/381a803ca3e2428db7752680e6ae1d58 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:14.945959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:14.960313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0fcbc-8804-4de2-ae9b-0d5cbec826f7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.005037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.006013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.006339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.006622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.007386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:15.007696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.009507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.021764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 220/432] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:15.022659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.069209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36bfa-8a42-4112-93fb-afe9bb9aa3a4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.106251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.107046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.107401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.107743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.108660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=381a803ca3e2428db7752680e6ae1d58) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:15.109044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.111149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.123171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 213/433] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:14 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=4642f59bea334153a5500859e246a0c6 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:15.124342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.143168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4adda-e44e-4d96-a7f5-4a481a8e21a3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.185131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.186026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.186339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.186835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.188971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.190928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.192840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 221/434] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:15.198759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.332561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe75b23-0539-47da-8b98-036fd7001d3d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.378875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c89626-6964-4fa5-9997-2ed3c98224b8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.396082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.396870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.397225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.397576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.422107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.423605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.423902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.424182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.425034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=381a803ca3e2428db7752680e6ae1d58) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:15.425347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.427693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.434866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 222/435] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=381a803ca3e2428db7752680e6ae1d58&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:15.486407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.488173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39009509-c634-4205-9474-fadfe9b1b585 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.606825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 214/436] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:15.615396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.643609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b728b2-38a5-42db-94b0-8a0133f0eb6e None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.644194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b728b2-38a5-42db-94b0-8a0133f0eb6e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.644687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b728b2-38a5-42db-94b0-8a0133f0eb6e None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.646960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 215/437] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:57:15.656266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.656618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.656909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.678957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:15.680665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:15.759647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa32e2-eeed-4ba2-8651-523af32c5b42 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.791947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 216/438] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:15.807949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.808824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.809111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.809388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.812160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:15.812509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:15.816684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 223/439] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:15.825150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.825323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:15.946417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55e0a-a2df-4ed3-a31d-c6a9a3e6da5f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.953781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342aa7d-982d-4200-bdef-ccc69235cba3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:15.988175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.988988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.989272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.989547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.996258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:15.997136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:15.997416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:15.997690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:15.998930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.000380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593603c0-a626-4ae4-8962-ebe2bc78093b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.001576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:16.003505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 217/440] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:16.004676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a74d-d724-425d-89e9-364c56446592 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.004795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a74d-d724-425d-89e9-364c56446592 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.005034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a74d-d724-425d-89e9-364c56446592 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.006899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 218/441] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:16.011678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570a74d-d724-425d-89e9-364c56446592 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-60a5b4de-864b-4316-ae3d-4ac0ea8f6a23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:16.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:16.013238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:16.014031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 224/442] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:16.017063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.017351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.017829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.042184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:16.043455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:16.128905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 225/443] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:16.130073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.151467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570a74d-d724-425d-89e9-364c56446592 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.185852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.186544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.186829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.187105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.187939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=88a0e9d0fe9d48eaae419a9d79677539, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:16.188251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.190592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.205282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 219/444] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=88a0e9d0fe9d48eaae419a9d79677539 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:16.206475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.249862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b5e40-05a6-403e-9370-daff00d7f641 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.290857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.291223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.291546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.291781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.292567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:16.292942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.294327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.305205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 226/445] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:16.311383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.331982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e39d-c4b6-4838-a47e-4f20ccacb9de None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.370448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.371266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.371546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.371819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.385939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:16.395408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:16.396372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:16.397165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:16.401455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 220/446] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:16.402480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.431149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f13c-4724-46d2-bed0-8f61c9ea17f6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.467354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.468006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.468289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.468561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.522517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620cf2-f58a-42c8-8e60-bdc2f02c5e3f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.543199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.544600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.566666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.568033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.568317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.568875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.569874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:16.571192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.572395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.581504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 221/447] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:16.587968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.662778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 227/448] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:16.668510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.695999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aef91-1890-4739-a5da-6a5438672a3a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.731799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.732451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.732726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.732998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.771547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb110960-6d45-40fe-9124-98db22315c70 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.801240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.802663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.805005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.806602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.806602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.806602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] PATH_INFO: `/v3/projects/fab88b77b7fd44528a3ca6841d9f273b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.807668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] RBAC: Authorizing `identity:get_project(project_id=fab88b77b7fd44528a3ca6841d9f273b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.809098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.810850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 228/449] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/projects/fab88b77b7fd44528a3ca6841d9f273b => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:16.817033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.925749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef09e5-9111-4eb6-b0f6-f7350ba1dd74 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:16.929109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 222/450] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:16.934728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:16.961883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:16.962783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:16.963055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:16.963343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] PATH_INFO: `/v3/groups/b00947654146476080a7f1c2141af0a2` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:16.964531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] RBAC: Authorizing `identity:get_group(group_id=b00947654146476080a7f1c2141af0a2)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:16.965986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:16.967645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 229/451] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/groups/b00947654146476080a7f1c2141af0a2 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:16.973974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:17.037107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f69ac-95ba-43f4-bade-2ae33e5d2e6f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:17.071472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:17.072252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:17.072540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:17.072815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] PATH_INFO: `/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:17.074016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] RBAC: Authorizing `identity:get_project(project_id=0a1c57e3b7564f7591a2ec7bb4ce5bcd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:17.074601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bb2e-998e-48f3-93e2-0348944bab56 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:17.075493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:17.077204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 223/452] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/projects/0a1c57e3b7564f7591a2ec7bb4ce5bcd => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:17.083350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:17.110061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:17.111224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:17.111368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:17.111535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:17.112362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] RBAC: Adding query filter params (group.id=b00947654146476080a7f1c2141af0a2, role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=fab88b77b7fd44528a3ca6841d9f273b) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:17.112668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:17.114106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:17.120043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 230/453] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:16 2022] GET /identity/v3/role_assignments?group.id=b00947654146476080a7f1c2141af0a2&scope.project.id=fab88b77b7fd44528a3ca6841d9f273b&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:17.207043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38ecd-f85a-4681-8c72-2e1e7388448b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:17.251690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:17.252644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:17.252934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:17.253605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] PATH_INFO: `/v3/groups/f4d11e65ad8449abafddae3343fabebf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:17.255257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] RBAC: Authorizing `identity:get_group(group_id=f4d11e65ad8449abafddae3343fabebf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:17.256797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbdbb6-50a0-4438-b94e-24821737aa07 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:17.258838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 224/454] 10.222.0.58 () {56 vars in 1180 bytes} [Tue May 10 13:57:17 2022] GET /identity/v3/groups/f4d11e65ad8449abafddae3343fabebf => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:17.265699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:17.375189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32724-1597-4af8-aca7-d506dcce034d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:17.408983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:17.409637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:17.409917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:17.410190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:17.411051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] RBAC: Adding query filter params (group.id=f4d11e65ad8449abafddae3343fabebf, role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:17.411363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:17.412800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55f881-6e14-4a05-aaa2-e58819dd90b3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:17.418921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 231/455] 10.222.0.58 () {56 vars in 1376 bytes} [Tue May 10 13:57:17 2022] GET /identity/v3/role_assignments?group.id=f4d11e65ad8449abafddae3343fabebf&scope.project.id=0a1c57e3b7564f7591a2ec7bb4ce5bcd&role.id=4642f59bea334153a5500859e246a0c6 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:20.574488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669e14-d852-442a-8501-0c1ebae34e1c None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.575017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669e14-d852-442a-8501-0c1ebae34e1c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.575017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669e14-d852-442a-8501-0c1ebae34e1c None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.576778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 225/456] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:20.583703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.584211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.584286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.611761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:20.613699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:20.676598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8366c9-75a1-49f6-974d-4608541f73e2 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.676966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8366c9-75a1-49f6-974d-4608541f73e2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.677462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8366c9-75a1-49f6-974d-4608541f73e2 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.679548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 226/457] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:20.685546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde085a-1fa6-4332-a2f1-202ff76f860a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.685856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde085a-1fa6-4332-a2f1-202ff76f860a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.686114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde085a-1fa6-4332-a2f1-202ff76f860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.715391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1bde085a-1fa6-4332-a2f1-202ff76f860a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:20.722573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1bde085a-1fa6-4332-a2f1-202ff76f860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:20.733204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 232/458] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:20.759025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:20.847774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 227/459] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:20.849086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688bd45-bfc4-4f58-babb-a043a0e1e438 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.849376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688bd45-bfc4-4f58-babb-a043a0e1e438 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.849655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688bd45-bfc4-4f58-babb-a043a0e1e438 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.857810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 228/460] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:20.873516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.873914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.874079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.878047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d46e8-65ca-489f-9ab5-a639954bae3e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:20.903172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:20.904622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:20.917604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:20.918784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:20.919099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:20.919385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:20.931319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:20.932804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:20.953033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:20.954162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:20.955726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:20.955726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:20.955726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:20.955726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:20.957365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 233/461] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:20.963245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:20.994756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 229/462] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:20.995514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.081205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a42a-e9a4-4f27-b379-470faffaca34 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.117571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.118644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.119127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.119648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.132063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.133999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.150102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65582-3395-4d26-bd19-9b4464458cd9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:21.154459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:21.155649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:21.156698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:21.156698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:21.156698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:21.156698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:21.159015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 234/463] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:21.160109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.202649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.203395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.203686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.204168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.205857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:21.206227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.208930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.222518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 230/464] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:21.228264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.277852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fcc9f-9431-4b0b-a16b-f7d771076f60 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.315179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.319951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.319951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.319951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.328668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.330539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.338840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e4510-a742-46b9-80e2-8a74202b8665 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:21.349717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:21.350812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:21.352644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:21.352644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:21.352644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:21.352644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:21.354596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 235/465] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:21.356326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.383650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.384520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.384930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.385214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.390148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:21.390148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.390148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.405031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 231/466] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:21.406183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.480367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b625d0f-9384-49e1-bda1-515c3abad638 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.518496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464393-b795-41ad-82b7-560b86b80e3c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.522382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.523071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.523357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.523633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.524434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.526026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae748cc-83e5-4798-9f2b-bc325af9a870 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.557842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.558540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.558849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.559124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.559911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:21.560218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.561585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc877-29d1-4e1d-a78b-5f0866de1814 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.574451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 232/467] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:21.575919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2c7e7-fb8b-4fb6-9f9f-37ebf87b37c6 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.576496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2c7e7-fb8b-4fb6-9f9f-37ebf87b37c6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.576496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2c7e7-fb8b-4fb6-9f9f-37ebf87b37c6 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.577895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 233/468] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:21.579428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2c7e7-fb8b-4fb6-9f9f-37ebf87b37c6 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.645205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 236/469] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:21 2022] POST /identity/v3/users => generated 246 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:21.647202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.647628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.647917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.674659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:21.676000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:21.721768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2c7e7-fb8b-4fb6-9f9f-37ebf87b37c6 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.771230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.771949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.772236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.772514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.773254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.774820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:21.776347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 237/470] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:21.777493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.838124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 234/471] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:21 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:21.843333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:21.906342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e4f73-d979-4136-9cd9-b0428fb6db56 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:21.961036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:21.961741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:21.962024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:21.962351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:21.963095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:21.964590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ad335-760a-4d06-9f07-ac65c6c5b79a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.002725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06649-6cd6-482a-833f-fadcf9025bdf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.046464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.047977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.048454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.048783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.061898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:22.063434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.075223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 238/472] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:21 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:22.077496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cff9-2066-4734-9247-2521d6d92287 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.081820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cff9-2066-4734-9247-2521d6d92287 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.081820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cff9-2066-4734-9247-2521d6d92287 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.081820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 239/473] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:22.087236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.087592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.087842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:22.088623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:22.089551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:22.090444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:22.090444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:22.090444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:22.090444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:22.093302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 235/474] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:22.099546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:22.112514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:22.114601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:22.214556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 240/475] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:22.242439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:22.266251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292da9-b49a-47d2-9a26-6d6618452515 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.308069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.308944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.309237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.309521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.310794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:22.311114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:22.312842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.334844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 236/476] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:22.341990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:22.380430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04d1a-6ec4-40d8-8efb-cae979d83633 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.415886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.416558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.416835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.417106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.417911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:22.419303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.459958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d19579-070c-4325-aafb-3c392fba7ad8 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.460314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.sql.core [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 13:57:22.460314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 13:57:22.460314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [parameters: {'id': '3180d46eac8f4a89845677f0c47bc896', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 13:57:22.460314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80414) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application conn.query(q) May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application result.read() May 10 13:57:22.482445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return ref.to_dict() May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:57:22.483353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application next(self.gen) May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application yield resource May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application next(self.gen) May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self.session.rollback() May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self.force_reraise() May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise self.value May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application session.commit() May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 13:57:22.484270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._prepare_impl() May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self.session.flush() May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._flush(objects) May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application compat.raise_( May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application flush_context.execute() May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rec.execute(self) May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 13:57:22.485113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:57:22.487263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application conn.query(q) May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application result.read() May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:57:22.488086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [parameters: {'id': '3180d46eac8f4a89845677f0c47bc896', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:22.488867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 10 13:57:22.489630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.491983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 241/477] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 10 13:57:22 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:22.491983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:22.515038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.516098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.516607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.517133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.518537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:22.520254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.601183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 237/478] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:22 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:22.614790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a3c0-158c-430f-bc4c-7258801a5554 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.677895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.678865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.679200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.679479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.693605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d930b110-8ad7-476b-b7ab-3dae8b18651e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:22.704479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:22.705742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 10 13:57:22.706562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:22.707156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 242/479] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:57:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:22.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:22.858735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a135720-fdfb-461c-8470-834abeded692 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:22.896151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:22.897177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:22.897655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:22.898141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:22.899103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:22.899634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:22.901212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c7993-01c5-4ba1-a24a-a5b71d1fe0d6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:22.911457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 238/480] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:24.350248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031c73-4225-41f6-ba8e-befd29d53cbd None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.350926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031c73-4225-41f6-ba8e-befd29d53cbd None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.350926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d031c73-4225-41f6-ba8e-befd29d53cbd None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.352501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 243/481] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:24.358224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.358860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.358860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.387370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:24.388596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:24.492044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 239/482] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:24.522704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d031c73-4225-41f6-ba8e-befd29d53cbd None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:24.663743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d031c73-4225-41f6-ba8e-befd29d53cbd None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:24.729421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:24.730266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.730552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.730821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.749154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8c8a8327-fe16-465d-94ff-489491298388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:24.757212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:24.758116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:24.758951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 244/483] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:24.765813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:24.855032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101ee06-4aef-4dee-a01c-a416d9422d9f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.855341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101ee06-4aef-4dee-a01c-a416d9422d9f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.855585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101ee06-4aef-4dee-a01c-a416d9422d9f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.857088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 245/484] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:24.864138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.864441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.864634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.902835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae013f5-f481-4388-a1cd-27faa8f642d6 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:24.907298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:24.909505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:24.943479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:24.944509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:24.945015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:24.945496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:24.946529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:24.947112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:24.948750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:24.959092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 240/485] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:24.967444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.060008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 246/486] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:25.087030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.170313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92b1b-5eb9-4093-b971-37895577eaa5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.208576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3ad47-68a4-469a-80a5-f5cbe803fa44 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.209012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.210405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.210805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.211120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.223354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.225653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1028ce07-aa0d-4c6b-9388-5888082d8ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:25.247966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:25.248871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:25.249711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:25.249711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:25.249711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:25.249711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:25.249711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 241/487] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:25.250970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795452d4-10a7-4a5b-81db-47cae3734541 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.251263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795452d4-10a7-4a5b-81db-47cae3734541 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.252151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795452d4-10a7-4a5b-81db-47cae3734541 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.262229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 242/488] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:25.262410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.262506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.262506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.262506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.262719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.262719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.262719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.279120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:25.279423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:25.280299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:25.287899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:25.289500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 247/489] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:25.292986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.376057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 243/490] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:25.377010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.427473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb223e5-4832-4c2b-8561-0a4ab49dbe65 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.465151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.465748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.466033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.466363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.467106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:25.467414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.468801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.486771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 248/491] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:25.487850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.494618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce0ab-b29a-4690-ba39-a0599016aa1a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.533607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.534446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.534737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.535016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.535781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:25.536094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.537492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.546550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 244/492] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:25.547480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.592130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58e1e-1b9b-4b08-ab2e-84a171ace9f8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.628619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.629700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.629981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.630279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.641123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:25.650382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:25.652498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:25.666500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:25.668417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38430a-444b-4c05-a5d8-fef5303fc355 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.668584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 249/493] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:25.668584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.693964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.694827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.695116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.695398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.704032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.713897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:25.722413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:25.724549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:25.727321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:25.728553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 245/494] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:25.735169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.776274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ff14-fa8f-42ae-9556-32b891b675bb None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.811975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.812749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.813032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.813307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.825780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.827310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:25.846429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:25.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 250/495] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:25.850577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.851330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d678-e30c-422f-abbd-5790af933ad0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.887747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.888423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.888699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.888971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.889715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:25.890021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.891443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:25.904673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 246/496] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:25.904673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:25.961119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840547-f125-42b2-88bd-a7745fd5d558 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:25.995760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:25.996351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:25.996634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:25.996909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:25.997754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:25.998069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:25.999728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23e86b-b143-4c77-b980-28f6d0555df9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.009776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f601db5-ccb5-4209-89d3-004a3480cb9a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.040072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 251/497] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:26.041402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946fe68-f562-4c5b-a474-2382d543a993 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.041697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946fe68-f562-4c5b-a474-2382d543a993 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.041969 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946fe68-f562-4c5b-a474-2382d543a993 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.043773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 252/498] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:26.044317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946fe68-f562-4c5b-a474-2382d543a993 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.049015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.049692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.049973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.050273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.051003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:26.051312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.052694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edf01b-29af-44b8-93d7-0fbe83759fb0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 247/499] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c68a-2c18-49b2-92cf-4d6c04842c5b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c68a-2c18-49b2-92cf-4d6c04842c5b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c68a-2c18-49b2-92cf-4d6c04842c5b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 248/500] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:26.069640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75c68a-2c18-49b2-92cf-4d6c04842c5b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.151321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946fe68-f562-4c5b-a474-2382d543a993 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.180738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75c68a-2c18-49b2-92cf-4d6c04842c5b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.186536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.187334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.187613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.187883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.198332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.199822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.216861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.217513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.217799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.218079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b34443b3-28fd-4d44-a093-7fc262b260cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:26.218677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.219556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.221144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:26.221144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:26.221144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.221144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:26.221144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 253/501] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.223905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.226142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.226459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.243119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:26.244304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:26.286520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.287874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097db11-3a90-4efb-b4cc-f33bd890b60f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.309928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 254/502] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:26.311114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.399623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 249/503] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:26.400590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.400873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.401147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.419204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:26.420404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:26.426964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed466f80-a27f-43eb-93dd-b5682e550212 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.460546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.461406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.461629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.461905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.469245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.471434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:26.487863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.489657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:26.490853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:26.491596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 250/504] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:26.491596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.491732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 255/505] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.496254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.590610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584b14-6957-48b1-9795-f0da55d959c5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.600819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654bc4-c25a-4c5c-8e8a-8762bf73479c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.623358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.624032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.624308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.624578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.625292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:26.625593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.627035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.634729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.635523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.635804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.636079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.638285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 251/506] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:26.639104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.647290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.648872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:26.668049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.668957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.669792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:26.669792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:26.669792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.669792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:26.669792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 256/507] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.670443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.757365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045b15-6bd7-410e-9b17-5865b02b1928 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.776230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889aa36-da9d-44cf-9d1a-86a1a56cd345 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.790511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.791308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.791597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.791867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.802817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.804221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.811128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.811800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.812086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.812366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.813212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:26.813524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.815693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:26.821893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.822941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.823833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:26.823833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:26.823833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:26.823833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:26.823833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 252/508] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.824100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.838526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 257/509] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:26.839290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.932541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafbd5-f32b-4eee-a041-466d4fa87b05 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.937726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201314a-7c12-47c6-a631-607c0c8bfd72 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:26.966357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.967158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.967441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.967722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.971676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:26.972464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:26.972746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:26.973022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:26.980253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:26.981435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:26.981731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:26.989803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.991548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 253/510] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:26.997334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:26.998468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 258/511] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:26.999957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.100194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880159a9-03f7-41f8-abce-cf472a0166d9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.110479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb289bf6-14c9-4ebe-bc3c-5423e92de5db None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.135128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.135801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.136081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.136360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.137084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:27.137391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.138805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.145796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.146505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.146788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.147129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.148096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:27.148406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.149800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.151039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 254/512] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:27.151888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.166265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 259/513] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:27.166265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.254781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9b7cd-ac81-4810-89da-3aec359698e4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.271971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff6e-19a3-464e-9df6-3abf04e90f81 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.288108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.288757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.289038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.289316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.306781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.307455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.307738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.308018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.308778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:27.309086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.310503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.318798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 260/514] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:27.319612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.355918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.357367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.422775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af0bfa-09f8-4b9a-aeb8-8ffcab2881cd None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.455751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.456437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.456716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.456986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.457830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:27.457966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 255/515] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:27.458482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.458833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.459596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.477906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 261/516] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:27.478800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.557018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33363a74-da7f-45ec-850f-1ce0a7b5d114 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.578105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b03b-87cd-487c-b97d-0b8d9d80979c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.589798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.590532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.591872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.591872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.591872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.593213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.614395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.615183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.615461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.616112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.622949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.624491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:27.640034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:27.644974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 262/517] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:27.646129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.706232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 256/518] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:27 2022] POST /identity/v3/users => generated 247 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:27.706232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.779353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c28a05-787e-4e3e-abb9-0d784d6fee03 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.817164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.817888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.818171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.818475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.827304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ea944-f704-4f1c-b704-90f01737ccc3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:27.830891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.832389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:27.852010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:27.852945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:27.853968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:27.853968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:27.853968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:27.853968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:27.853968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 263/519] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:27.854276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.874292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:27.875143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:27.876247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:27.876247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:27.887739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:27.889751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:27.910006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:27.910952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:27.911798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:27.911798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:27.911798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:27.911798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:27.911798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 257/520] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:27.913018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:27.977437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163523db-20c1-47c0-9915-283ae9cd131e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.013153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.013808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.014096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.014403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.030656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a730-a085-48fd-ab01-be76c4bcb0c9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.065699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.066402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.066689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.066969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.067821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:28.068081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.069509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.086977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.097112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 258/521] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:28.097399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.097550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.195240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 264/522] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:28.195928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.203718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54521-b7b4-4497-a33f-1f12205807cb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.240869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.241948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.242678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.243025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.243837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:28.244139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.245613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.257970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 259/523] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:28.258842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.320481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18af62-8a80-4c2f-a58a-fff2b098296d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.357092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.357804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.358084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.358388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.359112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:28.359423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.361514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.380162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 265/524] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:28.381445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.382407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144a3d-b419-4313-aea3-d696b01f15a6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.425329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.426008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.426321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.426651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.427496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:28.427815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.429457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.437441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 260/525] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:28.437441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.522116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af679f6-21f4-4ac0-8180-6b6b2da51122 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.553640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937fbfb-80ea-424d-86a9-17a66c474451 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.565270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.566321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.566655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.566933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.579170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.581057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.593994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.595331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.595862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.596366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:28.601003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:28.601958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:28.603307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:28.603307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:28.603307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:28.603307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:28.603307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 266/526] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:28.607795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.609066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.611593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:28.627345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:28.628423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:28.629601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:28.636640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 261/527] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:28.637886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.737569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d7d1-0958-4eba-bceb-448bb85855e3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.755152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899f360-f45e-4714-a753-bfeeab9a20c1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.786963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.788120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.788634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.789134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.793418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.794468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.794871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.795434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.796546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:28.796998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.798503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.799796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.800404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:28.817578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:28.818543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:28.819525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:28.822280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 267/528] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:28.822641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 262/529] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:28.824036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.825929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:28.947740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7732-7023-4080-a9c8-3fff42f99f0d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.955521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54124-766e-4287-858a-c2c3770db1b3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:28.988611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:28.989415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:28.989697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:28.989968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:28.997996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:28.999647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.001563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.003077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.004329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.004835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.006434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:29.006961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.008779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:29.016352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:29.018218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:29.019158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:29.023908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 263/530] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:29.024853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.034159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 268/531] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 10 13:57:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:29.035081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.136133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adedb-e884-4170-bd1b-c6553167b220 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.162105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11086-1851-4ac2-bb4f-504913b69b2f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.175755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.176423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.176704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.176978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.177954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:29.178516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.180655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.195412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 264/532] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:29.196352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.208381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.209553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.209897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.210183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.211380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:29.211705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.213877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.238254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 269/533] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:29.239138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.308791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd6d3e-bef9-4974-ad82-61cb375034fe None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.353481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.354154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.354621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.354903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.365086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a0621-727e-4863-91eb-990e63f68644 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.401671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.402545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.402902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.403304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.404238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=4ee88945608d42c0981bb93470e832fa) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:29.404626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.406210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.412825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 270/534] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/role_assignments?user.id=4ee88945608d42c0981bb93470e832fa&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:29.413751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.465085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.474711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.542150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cba9-e7f9-4adc-b65f-79f84af1ffb0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.588941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.589627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.589915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.590192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.591070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=0ef044e8d1fb4953b2cc3244f4a4369f) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:29.591386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.592862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.599640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 271/535] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/role_assignments?user.id=0ef044e8d1fb4953b2cc3244f4a4369f&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:29.623731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 265/536] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:29.631129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:29.783912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f573a-288c-49b1-a449-af808608773a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:29.828595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:29.829458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:29.829799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:29.830062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:29.847429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:29.849107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:29.873650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:29.875567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:29.877268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:29.877268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:29.877268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:29.877268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:29.877268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 272/537] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:29.883508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.005668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cccf4-0c87-442c-9ff2-9c3dd92542e4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.050975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.051667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.051953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.052232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.053037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:30.053349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:30.054812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e969-24be-414f-9201-75c07948d544 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:30.067579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 266/538] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:30.073304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.217338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb7c6-ea43-42be-a214-a944f0311951 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.226363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaf340-4086-4a1d-8b36-b9d27577b50f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.226675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaf340-4086-4a1d-8b36-b9d27577b50f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.226937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeaf340-4086-4a1d-8b36-b9d27577b50f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.228595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 267/539] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:30.241418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.241716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.241984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.264477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.265638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.266143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.267690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.275215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:30.277000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:30.279326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:30.280565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:30.292746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:30.293654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:30.294507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:30.301512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 273/540] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:30.303148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.418039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 268/541] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:30.457613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076d1a-2c2c-49c0-ba9d-b100c796165a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.474712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.503778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.504818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.505324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.505804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.506854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:30.507361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:30.508984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:30.539675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 274/542] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:30.554670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.607304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc3b74-77e6-467d-af2d-4c74abd44511 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.648601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.649472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.649761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.650038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.674792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:30.684312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:30.685263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:30.686073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:30.697498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 269/543] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:30.711177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.786562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b63f-5d3a-4242-8dee-49dbfe85537e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.850759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.851405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.851695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.851977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.852845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=cf1f369c503e48b08e3faa9738c5df8e) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:30.853166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:30.854713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:30.876960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 275/544] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/role_assignments?user.id=cf1f369c503e48b08e3faa9738c5df8e&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:30.877894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f05543-b035-4944-abb4-64929b09a6ba None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:30.966529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:30.967231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.967515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.967795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:30.968566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:30.968878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:30.971276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eadb63-51e8-4522-b4d8-0f3790560c52 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:30.982366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 270/545] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:30.987946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:30.998236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dfb72-7715-45b1-a48c-d6417c209beb None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:30.998559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dfb72-7715-45b1-a48c-d6417c209beb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:30.998805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dfb72-7715-45b1-a48c-d6417c209beb None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.003008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 271/546] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:31.007353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.007937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.008410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.027328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:31.029020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:31.141578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 272/547] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:31.150551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d95826-6fa4-46ca-a6c1-9e466913c7cf None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.180233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.227392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.228674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.228989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.229397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.260436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:31.261914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:31.295347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:31.296460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:31.297467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:31.297467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:31.297467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:31.297467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:31.306709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 276/548] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:31.315155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03518560-5478-44fc-98ae-7d90638bce68 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.316423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.355591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.356647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.357099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.357537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.369612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:31.379479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:31.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:31.381488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:31.383735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 273/549] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:31.391045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.497109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ba118-69ab-4d3a-8538-70703ad80559 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.535048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.535330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdab43-ffc2-4177-8e4c-095a86ffc13e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.536123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.536574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.537010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.538094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:31.538587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:31.540167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:31.549996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 274/550] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:31.555273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.596323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.597002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.597288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.597571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.598330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:31.598645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:31.600053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:31.628287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 277/551] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:31.635491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.676921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7a5f3-a1c8-4b2e-982a-1af833557b7a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.717348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.718607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.719900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.720455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.733001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:31.734713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:31.755821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:31.756793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:31.757775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:31.757775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:31.757775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:31.757775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:31.759460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 275/552] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:31.768391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:31.880455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3977c0-583e-4fa2-8da5-af7c76c5e24e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.927732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:31.928546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:31.928832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:31.929111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:31.943228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:31.944731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:31.969649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:31.970784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:31.971661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:31.972158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a97d-1407-408a-b423-349ad952ce08 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:31.979619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 278/553] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:31.987238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.012956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.013549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.013831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.014107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.014912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:32.015223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.017175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.031132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 276/554] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:32.036688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.140174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5fcc-bcb8-49d7-aab0-54d3d6d28d0f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.168535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fedceb-8a58-4b01-b1ad-a4403a80ba68 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.176962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.177775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.178058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.178370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.187095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.189629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:32.207258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:32.208232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:32.210359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 277/555] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:32.214486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.260860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.261977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.262497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.263056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.264151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:32.264695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.266989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.314325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 279/556] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:57:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:32.332364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.351604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16a12f-f953-4b54-863f-1b661fceb022 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.390676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.392170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.392754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.392956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.393837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:32.394135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.395636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.436111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 278/557] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:32.441935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.490001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c79ae2-338b-494e-9cad-cd8643008ad6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.528244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.528873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.529157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.529430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.557478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b187af4-9a0b-4724-874d-d754bf78de0a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.600562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.601705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.602598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.602879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.603153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.603338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5537e9-fbc8-4b38-b1f7-5499d9b8ba4f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.614067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.615631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-fc7b60b6-e7e5-45ac-8147-ec59227083de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:32.633505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:32.634541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 279/558] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278f14-944a-4a64-a6f8-8b7fb71dbec8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278f14-944a-4a64-a6f8-8b7fb71dbec8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d278f14-944a-4a64-a6f8-8b7fb71dbec8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.640403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 280/559] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:32.642168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84bab6-e930-4075-a1fb-064a785d7f09 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.642168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84bab6-e930-4075-a1fb-064a785d7f09 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.642168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84bab6-e930-4075-a1fb-064a785d7f09 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.643284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 281/560] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:32.643845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84bab6-e930-4075-a1fb-064a785d7f09 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.719469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 280/561] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:32.720297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167643-56f6-45a5-ae0d-85a37f8253d6 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.720591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167643-56f6-45a5-ae0d-85a37f8253d6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.720870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167643-56f6-45a5-ae0d-85a37f8253d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.737479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e9167643-56f6-45a5-ae0d-85a37f8253d6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:32.738746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e9167643-56f6-45a5-ae0d-85a37f8253d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:32.764030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84bab6-e930-4075-a1fb-064a785d7f09 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.803502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.804092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.804380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.804656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.805378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:32.805685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:32.807099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:32.807741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 281/562] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:32.809163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.809462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.809742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:32.831829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 282/563] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:32.831829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.836113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:32.836884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:32.918228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 282/564] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:32.918228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:32.954945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1de04d-50e3-450f-ae98-8184ed239c76 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:32.991369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:32.992160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:32.992441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:32.992716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.003169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.004885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:33.023802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.024885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.026011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:33.026011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:33.026011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:33.026011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:33.033791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 283/565] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.033791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.034069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8386-bdaf-4850-b407-452bdd2bdfc3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.065570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.066323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.066607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.066883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.074165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.075663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:33.090037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.091141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:33.092002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 283/566] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.092484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.141685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba4b20-5eda-43e0-9322-cc9398e7077b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.181472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.182395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.182710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.182976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.196424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:33.200450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54d2f-f005-49b6-b5a0-a90376a0abbe None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:33.204341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.205743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:33.206631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:33.210783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 284/567] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.211725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.236864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.237717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.238001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.238328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.249826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:33.258534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.259583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:33.260420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 284/568] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.261147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.334932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33490421-de0b-45af-8f4e-0a672765ed86 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.374641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.375412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.375660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.375938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.376664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:33.376972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.378658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.380998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a02077-de27-4c5d-9acf-810337f4259c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.391485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 285/569] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:33.393060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.418515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.419201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.419485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.419765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.420579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:33.420891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.423032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.435695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 285/570] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:33.436662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.509595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842ff3-09c1-4908-8023-87c99f28955e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.541797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ab2d6-98b9-4c7a-ac19-34d794906d46 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.555524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.556203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.556487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.556759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.557518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:33.557822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.559237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.570383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 286/571] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:33.571372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.581580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.582290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.582578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.582858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.583617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:33.583932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.585389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.595755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 286/572] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:33.596433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.680244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82a1c-8889-48a0-9952-c7048a1063f0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.708914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045175fd-591f-46f2-89a9-6ffea819b634 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.713364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.714157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.714487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.714760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.721958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.723328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:33.738519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.739459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:33.740333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 287/573] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.740986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.746295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.747155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.747444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.747720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.763294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=ffdd6ee2e1e64d2fb0d6d05687df82c1, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.764765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc350af-76a5-4362-8a14-6b55af07ae57 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.836549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 287/574] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:33 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:33.837867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6d1c7-8369-4236-8926-9388ea9f146f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.838164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6d1c7-8369-4236-8926-9388ea9f146f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.838495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6d1c7-8369-4236-8926-9388ea9f146f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.841036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 288/575] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:33.841036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6d1c7-8369-4236-8926-9388ea9f146f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.849173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ecd19-0ded-406a-98b5-fc7682b53a1d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:33.889636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:33.890469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:33.890804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:33.891271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:33.902799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:33.904273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:33.924427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:33.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:33.926292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:33.926292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:33.926292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:33.926292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:33.926292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 288/576] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:33.926582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:33.977001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6d1c7-8369-4236-8926-9388ea9f146f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.021308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.022235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.022513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.022781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.039584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.041070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.049762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f3df-d40d-4d65-9525-a73e7dabf0c5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d5e82587-36ff-4dd4-be74-60ebdfc33596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:34.062123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:34.063224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:34.064250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:34.064250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:34.064250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:34.064250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:34.064250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 289/577] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:34.065821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.066267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.066689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.084207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:34.085386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:34.085983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.086751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.086986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.087260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.088128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.088409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.089951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.103517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 289/578] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.104201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.169805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 290/579] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:34.169805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.231253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b55a84-b6b9-45dd-82d0-7d226321fc11 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.270607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.271859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.272149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.272680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.273413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.273718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.274335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0102b-ffad-4546-951b-f99b0f7b8e54 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.275138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.294359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 290/580] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.295269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.317694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.318408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.318692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.318966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.319697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.320006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.321387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.334187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 291/581] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.335068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.412514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5c253-335e-40bd-891a-82139b4de553 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.444797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf23b09-3452-4403-836d-4c9a51c5d121 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.459589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.460257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.460542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.460816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.461674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=62ef235951c84f62b91e30d56860a904) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.461970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.463466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.474240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 291/582] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/role_assignments?user.id=62ef235951c84f62b91e30d56860a904&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.474240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.490571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.491505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.491797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.492076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.504719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:34.513828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:34.514869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:34.515767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 292/583] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:34.516719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.596801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08857e1-a60e-479a-8e8c-b8b7c5b26dc4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.634973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.635798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.636077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.636350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.638188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4a4c-64ec-4b3b-ada3-811c3359df86 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.645101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.646601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:34.663744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:34.666101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 292/584] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:34.668455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.682958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.683872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.684186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.684464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.693591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.695266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:34.711943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:34.713704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 293/585] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:34.714682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.785584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ebd8-c2f7-4c7e-88d6-5464be7a36ec None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.822583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.823266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.823550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.823825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.824584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.824891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.825981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7330e-43d2-466d-a087-897993a00ad7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.826294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.840023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 293/586] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.841029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.864042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.864847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.865138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.865415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.866470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.866794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.868274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:34.891629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 294/587] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:34.892724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:34.945620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc30e-1166-4c47-9f10-fc219d0b0b0b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:34.980664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:34.981351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:34.981638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:34.981917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:34.982826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:34.983143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:34.985246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.001995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662bdcb-76b8-4328-873b-8539b68e33e8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.004015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 294/588] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.004876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.038863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.039665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.039949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.040224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.052322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.053788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:35.073329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.074349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 295/589] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.076647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.116303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25fd2-cf89-49d6-9eec-09223938d2a4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.153499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.154369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.154659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.154938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.166543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.168012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:35.186716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.189094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.191438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:35.191438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:35.191438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.191438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:35.191983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97784733-218a-4e31-86b9-3a96f954602e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.192137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 295/590] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.192137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.227803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.228499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.228802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.229061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.240539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.242042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:35.261009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.263232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 296/591] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.266166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.301632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99fcba-a2de-4ae3-862b-ac23bdb04f39 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.345831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.346580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.346932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.347211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.348128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:35.348441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.349965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.367168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 296/592] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.368132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.369079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ee4ba-327d-422d-b5cc-65390d68dd4f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.405442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.406157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.406482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.406736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.407478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:35.407787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.409177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.426382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 297/593] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.427377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.473233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3cc4f-fb4a-4cfd-ad68-2f2f6156dc0a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.507427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.508021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.508304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.508583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.509433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:35.509740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.511142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.524406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 297/594] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.525312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.530378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad362fd-dca9-46ed-80ec-8981223446c9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.566085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.566935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.567219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.567494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.575863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.577328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:35.592483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.595384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 298/595] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.597882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.637986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c426239-dc5f-4aa2-a9ec-528c1c0af4f4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.683518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c632060-133f-4613-83df-484f527f5322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.684616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c632060-133f-4613-83df-484f527f5322 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.684902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c632060-133f-4613-83df-484f527f5322 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.685494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c632060-133f-4613-83df-484f527f5322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.695487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c632060-133f-4613-83df-484f527f5322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.697587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c632060-133f-4613-83df-484f527f5322 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.701556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14060-2ff4-437a-ac0e-9092a6e3dcb4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-9c632060-133f-4613-83df-484f527f5322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:35.729057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.730144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:35.731078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:35.733690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 298/596] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.734891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c632060-133f-4613-83df-484f527f5322 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.760625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.761597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.761715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.761977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.770507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.773096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:35.791399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:35.792468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:35.793384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 299/597] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:35.794053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.873036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c632060-133f-4613-83df-484f527f5322 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.915530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.916596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.917221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.917657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.918078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a82a60-52e7-466e-90f7-7c3dcbc82ef6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:35.919884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:35.922346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.928279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.951847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 299/598] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.957635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:35.968092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:35.969094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:35.969755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:35.970179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:35.971548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:35.972008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:35.974268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:35.990637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 300/599] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:35.992070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.070162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2e3d-b09f-4509-b03a-ee7b04ff90c9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.107226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.108071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.108474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.108857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.110498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:36.110926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.112456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.120128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd07ae-36b5-42b9-8d74-aee98184191a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.125529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 300/600] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:36.126588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.158945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.160078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.160520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.161016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.173888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.175894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:36.195227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:36.196186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:36.197061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:36.197061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:36.197061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:36.197061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:36.205119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 301/601] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:36.206441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.231295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30eab0-ea42-4c11-b69e-e41ee29dd181 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.267827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.268757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.269041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.269354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/0ef044e8d1fb4953b2cc3244f4a4369f/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.285560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=0ef044e8d1fb4953b2cc3244f4a4369f, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.286829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.322300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751acc34-2d6e-4239-ae13-13d4d5dbeaf2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.355950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.356844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.357133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.357407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/4ee88945608d42c0981bb93470e832fa/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.372554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=4ee88945608d42c0981bb93470e832fa, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.374007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83793bac-f040-47c9-95b3-3aa089d0c2ae None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.390656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 301/602] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:36 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/0ef044e8d1fb4953b2cc3244f4a4369f/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:36.391990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.486853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 302/603] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:36 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/4ee88945608d42c0981bb93470e832fa/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:36.488297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3138b7-381a-4dc3-b72a-e1ebeb99c9e2 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.488591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3138b7-381a-4dc3-b72a-e1ebeb99c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.488868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3138b7-381a-4dc3-b72a-e1ebeb99c9e2 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.490750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 303/604] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:36.508268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.508717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.509089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.541511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335124d-094f-40be-a9e1-7734fcd76ca1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.564435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:36.565923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:36.597991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.599973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.600909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.601409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.602990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:36.603581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.605483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.626619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 302/605] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:36.637574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.691510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 304/606] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:36.725133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.817895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c434-9b65-410b-ba6f-04366a7a42ab None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.862315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.863062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.863347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.863622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.864350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99675173-1942-4bd0-973a-e7cfe548b79e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:36.887479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:36.888956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:36.913530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:36.914526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:36.915403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:36.915849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:36.917415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:36.917415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:36.917415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:36.925009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 303/607] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:36.931868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:36.939037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:36.946987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:36.947951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:36.950621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:36.954798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 305/608] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:36.959164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.088363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fb34c-4326-4e2e-b0ec-55db4a9c5c0d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.098382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2c2e4-c522-4676-b68d-f297a09c33f6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.137955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.139191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.139674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.140733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.141695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:37.142005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.143471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78b7a5-77be-4120-957f-69282a52d05a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.144185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.145197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.145696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.146228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.147188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:37.147862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.149446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4102a11-8524-465e-85eb-85f877142165 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.161759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 304/609] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:37.161759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008d0c-a921-4e2c-a783-25d89280ac75 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.161759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008d0c-a921-4e2c-a783-25d89280ac75 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.161759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008d0c-a921-4e2c-a783-25d89280ac75 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.161759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 305/610] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:37.162764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5008d0c-a921-4e2c-a783-25d89280ac75 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.165016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 306/611] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:37.166714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.167225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.167709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.186123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:37.188388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:37.264819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 307/612] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:37.266461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.292324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5008d0c-a921-4e2c-a783-25d89280ac75 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.339857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.341014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.341293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.341570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/cf1f369c503e48b08e3faa9738c5df8e/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.382897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=cf1f369c503e48b08e3faa9738c5df8e, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.385316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.406498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f105b54-5a6b-4362-85d2-887d41393614 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.441896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.442907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.443299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.443681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.454872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.456525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:37.475506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:37.480135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:37.481084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:37.481084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:37.481084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:37.481084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:37.481390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 306/613] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:37 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/cf1f369c503e48b08e3faa9738c5df8e/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:37.481390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.481520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 308/614] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:37.482642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.681417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f80fe0-4f24-46bd-860a-a885b87e2201 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.720371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7b614-255d-453c-820d-1c497d911898 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.751006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.751822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.752103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.752378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.773727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:37.776127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.777090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.777628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.778100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:37.778712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:37.779614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:37.780708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 307/615] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:37.782010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:37.782526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.784101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.786041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.802879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 309/616] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:37.806859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:37.924607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ff849-afcc-4be8-9ab0-7bb4bcac0b28 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.939453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad35b-80c7-4bfc-82d0-905c410588e0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:37.962677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:37.963801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:37.964240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:37.964732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:37.973337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:37.975063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:37.990574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:37.991714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:37.992747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:38.000397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 310/617] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:38.002021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.006980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.007680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.007967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.008247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.009022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:38.009342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.010800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.028269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 308/618] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:38.034457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.113645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19ee3f-e8e8-4ae5-b2c1-beeb39ef4bf9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.149302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.150333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.150821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.151268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.152279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:38.152748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.154366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.174067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e018c0-1e0b-4551-b11d-9d04ff16aa8d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.179883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 311/619] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:38.181413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.216054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.216800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.217090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.217405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.235629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.237221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:38.255683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:38.256780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:38.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:38.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:38.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:38.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:38.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 309/620] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:38.262767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.298437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c7cde-2469-4188-b89c-69f22b5419d4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.340284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.342650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.343150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.344376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.417390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd58b7-3fb1-4fa1-997f-c073ad11413a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.472088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.472908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.473030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.473246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.473986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:38.474364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.475716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.480750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.482483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.496679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 310/621] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:38.502998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.602412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 312/622] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:38.604067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.707781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c601f-168b-4ffe-8c01-04d35041c3b2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.746610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9084b4-b3b2-4362-9645-f38bb6e265eb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.748006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.750296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.751045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.751664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.763176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.765518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:38.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:38.785940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:38.786954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:38.786954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:38.786954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:38.786954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:38.793571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 313/623] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:38.795256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.840308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.841216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.841595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.841938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.860845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.863477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:38.902520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:38.903603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:38.905007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 311/624] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:38.909965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:38.911295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120a611-9fff-4db6-aaef-2458c4127efe None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:38.948100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:38.949621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:38.950166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:38.950657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:38.951582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:38.952075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:38.954672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:38.972715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 314/625] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:38.974509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.105448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba21f2-2fbb-4bc8-9d47-3841cf2e66b5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.146668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.147957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.148490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.148992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.157890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.159957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:39.176585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:39.177544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:39.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:39.186314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 315/626] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:39.188534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.204220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629618c9-4707-4616-b24f-5019f5c77ee6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.268199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.269228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.269917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.270891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.272192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:39.272656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.274460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03283c-44e3-4b1d-b11a-9cd8844d4e3b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.298803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 312/627] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:57:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:39.302797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e98c2-9e86-44cb-90f1-be9f3f2c985f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.303084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e98c2-9e86-44cb-90f1-be9f3f2c985f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.303364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e98c2-9e86-44cb-90f1-be9f3f2c985f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.305094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 313/628] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:39.305939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a51db-b59d-4053-b3bc-877f3d83a385 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.306357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a51db-b59d-4053-b3bc-877f3d83a385 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.306691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a51db-b59d-4053-b3bc-877f3d83a385 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.308169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 314/629] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:39.308775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a51db-b59d-4053-b3bc-877f3d83a385 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.326059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af691ad-b99b-47b2-8c5e-ba951be845a2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.363860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.365209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.365675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.366447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.367751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:39.368270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.370716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fcfe1-2b75-47b7-9984-a751685ed4a5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.392560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 316/630] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:39.392560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5a17-1ea8-42d5-867a-20149696cb2c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.392560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5a17-1ea8-42d5-867a-20149696cb2c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.392560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5a17-1ea8-42d5-867a-20149696cb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.406840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-490e5a17-1ea8-42d5-867a-20149696cb2c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:39.408044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-490e5a17-1ea8-42d5-867a-20149696cb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:39.441458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a51db-b59d-4053-b3bc-877f3d83a385 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.480897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.481700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.481979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.482290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.489321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 317/631] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:39.491010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.491480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.492014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.502343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.504148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.513632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:39.516737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:39.525876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:39.526836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:39.527759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:39.527759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:39.527759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:39.527759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:39.530848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 315/632] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:39.532394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.602243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 318/633] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:39.603122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.647440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83800b7-db79-49ec-8fb8-2a7654693da3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.685013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.685677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.685985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.686257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.687748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:39.688134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.689585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.700130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 316/634] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:39.700999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.751915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1ea52-3f40-496c-a7be-6d14d25f9bb2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.803198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.804687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.805358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.806327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.826526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:39.832843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e80ac-692c-4781-8a87-365b4875ce74 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:39.837650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:39.838859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:39.839858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:39.842478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 319/635] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:39.844528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.876075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:39.877227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:39.877607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:39.878000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:39.878919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:39.879365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:39.881005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:39.900424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 317/636] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:39.901573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:39.986031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc11e6b-d539-4f96-8d82-261582094f34 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.027934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.028741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.029023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.029296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.045851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70804652-5ee6-4736-af77-c98345ed8430 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.047273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:40.055562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:40.056523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:40.057758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 320/637] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:40.058628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.099029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.099968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.100310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.100596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.101525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:40.101835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.103890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.115643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 318/638] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:40.116523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.208355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517ecf-4836-4557-a2ae-f6b995e2a1c5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.241294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302af161-02ae-4379-8baa-cf9ab10be6bf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.254236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.255489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.255947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.256368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.263832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.266404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.280917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.281702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.281862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.282142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.282923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:40.283231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.284605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:40.285163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:40.286183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:40.287177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:40.293682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 321/640] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:40.294862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.298229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 319/640] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:40.299608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.411179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b574e-7bfa-496e-9570-8e4683e7252d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.422583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b6b8-c447-4aca-9ce0-b068bee72c6b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.448787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.449589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.449881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.450140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.461430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.462925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.463818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.464483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.464885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.465126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.465843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:40.466161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.467680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-40737c72-af7b-4432-93b0-56465fa5006b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:40.487665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:40.489028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:40.490327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:40.490327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:40.490327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:40.490327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:40.490592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 320/641] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:40.490592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.490741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 322/642] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:40.490741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b307-d7f2-4bff-8c29-b45fe09f305b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.490741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b307-d7f2-4bff-8c29-b45fe09f305b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.490741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b307-d7f2-4bff-8c29-b45fe09f305b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.490741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 323/643] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:40.494306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b307-d7f2-4bff-8c29-b45fe09f305b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.610048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b307-d7f2-4bff-8c29-b45fe09f305b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.610356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cd0d-b605-4dfa-8ddf-cbe0141f0de9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.647503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.647923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.648185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.648471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.648736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.648834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.649068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.649291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.649470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:40.649818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.651197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.673634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 324/644] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:40.673634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.674021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.674021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-70656bf4-4abc-4951-9543-ca416b7af62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:40.691685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:40.693338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 321/645] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.695184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.718490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:40.719764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:40.788541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac4b0e-1e3f-42cb-abc6-1ee2ccf85326 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.802228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 322/646] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:40.802228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.827690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.828596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.828879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.829164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/62ef235951c84f62b91e30d56860a904/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.844724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=62ef235951c84f62b91e30d56860a904, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.846183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:40.917726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575667b-6041-43cd-8418-827a8f5cbeb7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:40.942329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 325/647] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:40 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/62ef235951c84f62b91e30d56860a904/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:40.942329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:40.974873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:40.975699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:40.975982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:40.976260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:40.994153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:40.996044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:41.019639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:41.020678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:57:41.021622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:41.022718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 323/648] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:41.026315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.062173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd2478-c353-46f8-94f4-2ca6a001151f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.100059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.100783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.101068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.101346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.102118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:41.102466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.103958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.122074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 326/649] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:41.122074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.179448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b572a3-e5f7-49cf-8869-88842969eea9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.231419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.232412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.232695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.233661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.237448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45392ada-c559-4d50-bddb-cc93c2eb68e8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.247771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:41.258292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:41.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:41.260104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:41.260740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 324/650] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:41.266224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.287789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.288678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.288966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.289245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.290099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:41.290923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.292713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.318508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 327/651] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:41.323154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.381838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c2adf-b4ee-4e9c-bb45-0020c8e0c2c6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.427185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.427768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.428050 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.428321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.439325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.440859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.453700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73766-787d-4a1b-9b6d-0e779ac9283e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:41.457870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:41.458893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 10 13:57:41.459846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:41.464406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 325/652] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:41.465391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.512412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.513458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.513949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.515780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.516866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:41.517479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.521013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.538181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 328/653] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:41.539747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.601800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ff2a2-b8de-4672-bfeb-6f07749b553f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.641436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.642345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.642478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.642686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.671004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da85c9-6480-4235-ae82-a0a734bcbf1d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.716965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.718869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.719342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.720624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.721746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:41.722283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.724667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.726130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.728741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.749450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 329/654] 10.222.0.58 () {56 vars in 1137 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:41.750826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.851165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 326/655] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:41.851165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:41.888561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e5662-3e87-4b99-8800-c6d112214869 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.935114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:41.936423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:41.936894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:41.937370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:41.951351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:41.953724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:41.969527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4d87-a037-4e5d-ba51-9dd5addc3a9a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:41.977916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:41.978935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:41.979839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:41.979839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:41.979839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:41.979839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:41.981889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 330/656] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:41.984038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.011112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.011995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.012385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.012763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.092375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.094833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.116001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9a19f-b14f-42cc-819f-8d6507d18b74 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.156440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.157515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.157911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.158330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.170128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.172620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:42.194147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:42.195295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:42.196437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:42.196437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:42.196437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:42.196437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:42.203037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 331/657] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:42.204162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.236695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 327/658] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:42.237750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.324661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee86d9-9c59-4685-a27c-97048818861e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.360849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.362414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.362414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.362414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.362792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:42.363105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.364547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbf73-ce61-4dc2-afb9-5b648ca94e29 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.375726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b3e0-9a4b-4913-82a6-a84d57ff2fbd None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.384556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 332/659] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:42.384556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75c7d-3408-434e-8845-be5c8ed525a1 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.384556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75c7d-3408-434e-8845-be5c8ed525a1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.384556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75c7d-3408-434e-8845-be5c8ed525a1 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.390069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 333/660] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:42.390069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75c7d-3408-434e-8845-be5c8ed525a1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.415637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.416413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.416698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.416972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.417701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:42.417996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.419489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.432936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 328/661] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:42.435539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.507805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75c7d-3408-434e-8845-be5c8ed525a1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.545047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.546785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.546785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.547078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.561129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.562380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07978795-1022-4331-899c-f5c8d9167ad2 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.562752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8708e219-c553-4df9-a5e5-d89474c96ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:42.584190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:42.585437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 334/662] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:42.586839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.587164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.587245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.612300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:42.613536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:42.615964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.616920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.617282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.617621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.628378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.629931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:42.644944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:42.646081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 10 13:57:42.647245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:42.652811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 329/663] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:42.653817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.692499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 335/664] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:42.693204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.794558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef213e7-5736-4b81-8680-9e510ff54f92 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.808284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4109-0074-4b8c-b81f-695d9ebd99cf None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:42.834452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.835273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.835558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.835834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.843763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.845228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.853883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:42.854889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:42.855374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:42.855824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:42.856784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:42.857220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:42.859539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:42.860507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:42.861495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:42.862548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:42.867001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 330/665] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:42.867771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:42.883454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 336/666] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:42.883454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.001585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90176d-c3d8-4dda-9e12-b24f65d34018 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.003015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674748-5cd6-4a92-8de1-8b7036612f15 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.037893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.038852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.039315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.039734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.040740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:43.041197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.042806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.044151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.044954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.045265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.045515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.056339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:43.064935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 331/667] 10.222.0.58 () {56 vars in 1139 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:43.065276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:43.066169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:43.067117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:43.068500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.070479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 337/668] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:43.071625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.179103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18103e2b-f8c1-484c-b09e-ee39e82b181b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.182059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a2059-0477-4ae8-a6a6-e8f349edc9c0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.223862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.224541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.224824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.225099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.225894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:43.226308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.227663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.228707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.229707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.230219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.230504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.238580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.239977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.240600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 332/669] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:43.241526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:43.266175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:43.267957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:43.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:43.272101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 338/670] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:43.273809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.365435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e0e89-93b8-4b71-a9de-7cb735cbccbf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.390059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9c6aa-1764-4fe4-843b-67b24ebfd59c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.400544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.401194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.401471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.401745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.428028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.428710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.428997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.429326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.430030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:43.430368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.431739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.441244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 339/671] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:43.442089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.473423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.474904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.542453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f40e-832b-4ce2-9cdd-d839b441972e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.576659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.577580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.577872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.578148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.579020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=0ef044e8d1fb4953b2cc3244f4a4369f) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:43.579333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.585439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 333/672] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:43.585632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c39621-a5fb-4e48-a8b2-e2719a01a59d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.585750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 340/673] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/role_assignments?user.id=0ef044e8d1fb4953b2cc3244f4a4369f&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d15b3-fec4-40ab-9f7e-b4e60b79789c None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d15b3-fec4-40ab-9f7e-b4e60b79789c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d15b3-fec4-40ab-9f7e-b4e60b79789c None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 341/674] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:43.598232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d15b3-fec4-40ab-9f7e-b4e60b79789c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.741355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fafb-836e-4e1a-87ba-f8595cc9601a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.744900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d15b3-fec4-40ab-9f7e-b4e60b79789c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.784010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.784684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.784961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.785232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.786025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:43.786373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.786542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.787617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.787836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.788010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.788438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.804592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.805085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 334/675] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:43.805960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.806384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-a6744563-3276-40b3-932d-66fc81f9c482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:43.828692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:43.829713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:43.830818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:43.830818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:43.830818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:43.830818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:43.832450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 342/676] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:43.833868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.834323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.834754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.852375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:43.853802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:43.925837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb13d2b-289b-4cb9-8c00-1b7bc34a9cbd None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:43.926829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 343/677] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:43.927854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:43.971765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:43.972627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:43.972858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:43.973135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:43.984323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:43.985778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:44.007422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:44.008496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:44.009402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:44.009402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:44.009402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:44.009402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:44.013169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 335/678] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:44.013917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.040165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5949c0a-f61c-4454-a813-14a5c560d636 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.080525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.081201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.081486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.081767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.082636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=4ee88945608d42c0981bb93470e832fa) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:44.082952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.084416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.094245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 344/679] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:43 2022] GET /identity/v3/role_assignments?user.id=4ee88945608d42c0981bb93470e832fa&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:44.094245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.142775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604585-b623-4e39-87db-18437fd5440b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.192523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.193601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.194098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.194599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.195544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:44.196057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.197709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.213423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 336/680] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:44.214674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.241722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f43b-9a9a-4ba2-ac22-45268f1d434a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.292959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.294556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.295586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.296014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.313035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:44.321265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:44.323054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 345/681] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:44.324536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.355567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf26d7-0315-4ffc-8329-0805cbeb6518 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.411769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.412536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.412717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.412987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.413707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:44.414009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.415405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.429240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 337/682] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:44.430384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.467481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a556f43-02c1-4efc-8e8d-783eccb2bd31 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.514551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.515773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.516235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.516686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.526561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.528497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:44.549589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:44.550760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:44.551886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:44.553388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dc53-59e0-4838-acc9-17d26481d3ef None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.554375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 346/683] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:44.555350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.593364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.594185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.594581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.594950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.596048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:44.596852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.599282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.611938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 338/684] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:44.613086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.702615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a5d-e72f-426a-9511-d5d741119580 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.744940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9400c-c51e-4bd0-b74b-ac10c9710c36 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.770882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.771658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.771938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.772317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.779663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.781170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.792565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.793586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.794095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.794787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:44.796664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:44.798233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:44.800103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:44.800733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:44.802179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.802898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 347/685] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:44.803900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.804880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.831493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 339/686] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:44.833396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:44.913964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b2185-f668-4b53-a0c0-bbd5a228229d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.952878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:44.953998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:44.953998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:44.954324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:44.960412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f01c7e-9e52-4fa4-b060-0e467aea843d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:44.965185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:44.966767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:44.990038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:44.991017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:44.991994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:44.991994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:44.991994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:44.991994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:44.992633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 348/687] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:44.993281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.001679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.003080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.003566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.004050 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.005067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:45.005613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.007345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.021075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 340/688] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:45.022254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.107447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27087-217f-463c-89b4-86ac0fc3c7ed None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.133629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c888d5-a3ce-48cd-85e7-77ccce6bbf34 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.148813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.149483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.149764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.150040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.175261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.176423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.177138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.177931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.179569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:45.180321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.182973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.196191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 341/689] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:45.197443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.226786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.228284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.301612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d901050-de7a-4fb7-87e2-de06924afd14 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.332066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 349/690] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 10 13:57:45.333107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.344269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.345318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.345820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.346342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.347380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=cf1f369c503e48b08e3faa9738c5df8e) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:45.347912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.349544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.362387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 342/691] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/role_assignments?user.id=cf1f369c503e48b08e3faa9738c5df8e&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:45.363733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.453756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5006e2-fae2-4810-96bd-12b450a1c9ce None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.508559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.509552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.509837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.510110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.522725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.524295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.527793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb7dad-5051-43a3-b4ee-d104070ae01b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:45.561291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:45.562328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:45.563250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 350/692] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:45.569158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.576787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.577635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.577924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.578218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.601095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.602614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:45.646107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:45.648319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:45.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:45.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:45.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:45.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:45.650247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 343/693] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:45.655221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.748358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5637-bbeb-46ae-b9d9-ad56a52ec53d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.790388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.791544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.791544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.791718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.792469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:45.792788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.794301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.809713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11ee1-cfbc-40e4-93ee-42bb7959f390 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:45.816291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 351/694] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:45.821554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:45.865949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:45.865949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:45.865949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:45.866536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:45.866890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:45.867160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:45.869588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:45.891485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 344/695] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:45.897212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.012692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb42f26-2835-46fa-b21c-b36c97cdbd2d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.057632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.058330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.058787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.059069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.078407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85105f-4cc6-4eb9-8f85-d2403f168abf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.115690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.116491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.116772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.117045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.126307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.128117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.143861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b4b624be-74e9-4208-8f84-39395c8a9cdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:46.144090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:46.144907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:46.145817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:46.146354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484868b-d90b-49a4-9df6-f99e7bbcf267 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.146424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 345/696] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:46.146993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa3e1-2ec6-47d2-8b0a-b0d4df6ba489 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.147195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa3e1-2ec6-47d2-8b0a-b0d4df6ba489 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.147367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa3e1-2ec6-47d2-8b0a-b0d4df6ba489 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.153000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 346/697] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:46.153950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fa3e1-2ec6-47d2-8b0a-b0d4df6ba489 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.266265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 352/698] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 10 13:57:46.267674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.267960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.268233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.285536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fa3e1-2ec6-47d2-8b0a-b0d4df6ba489 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.289017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:46.290256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:46.330906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.331605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.331883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.332157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.332954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:46.333261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.334846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.353189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 347/699] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:46.354659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.373401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 353/700] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:46.374311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.484925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca229-84bf-427d-9e7d-9d8f6be75c97 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.503465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3c293-91f3-43c3-a8bb-02a61138bc34 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.524557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.525389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.526894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.526894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.538556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.540141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.559218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.559896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.560184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.560582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.561527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:46.561853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:46.562863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:46.564596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:46.566732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:46.566732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:46.566732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:46.566732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:46.567257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.567403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 348/701] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:46.567403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.571536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 354/702] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:46.578600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.685054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a5a2-6c24-49d4-90dd-52962d7878c0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.692186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e299072-94e2-4e1a-bc1b-17ff3c362618 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.736810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.737637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.737915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.738278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.738951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.740415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.743542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.745060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.745602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.746370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.747385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:46.748949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.751236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4e7e4-4097-48b4-873d-b6b6728d93cb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-bdeed516-2668-45ae-8fa7-bb412c82b473 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:46.758127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:46.759210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:46.760214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 349/703] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:46.767566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c5a8-1f34-4512-ac8c-31239847961a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.767963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c5a8-1f34-4512-ac8c-31239847961a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.768354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58c5a8-1f34-4512-ac8c-31239847961a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.770085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 350/705] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:46.770284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 355/705] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:46.775137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58c5a8-1f34-4512-ac8c-31239847961a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.776960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.777475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.777954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.805275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:46.806568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:46.898302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 356/706] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:46.899568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:46.926656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58c5a8-1f34-4512-ac8c-31239847961a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:46.975071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:46.976562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:46.977279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:46.977997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:46.979492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:46.980287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:46.984012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:46.992735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 351/707] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:46.994494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.028650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d5e5c-54b4-416c-8031-088d6e9b93c3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.068162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.068940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.069226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.069500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.077435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.078972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:47.094036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:47.095116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:47.095999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:47.099961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 357/708] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:47.100920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.118436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987092aa-cb2d-4977-8ef6-5cb6642687a5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.155322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.156395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.156825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.157248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.168660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.170603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:47.188711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:47.189855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:47.191158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:47.191158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:47.191158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:47.191158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:47.193059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 352/709] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:47.194018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.214688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b1c8f-6073-4249-ac3f-2af8b638358d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.250645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.251430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.252068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.252437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.253261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:47.253663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.255196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.266510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 358/710] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:47.267900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.304062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634d86-008b-46c1-a501-c4c5bb89bc22 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.359325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.360419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.360729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.360972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.361792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:47.370246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.370246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.379450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 353/711] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:47.386263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.472917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ece90a-15c6-413d-be53-c486959a97b9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.508625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c88767-8bde-424a-a90a-5c87973c8800 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.515046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.516146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.516637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.517111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.518076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:47.518630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.520302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.547711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.548385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.548649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.548852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 359/712] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:47.549038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.549903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=62ef235951c84f62b91e30d56860a904) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:47.550243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.551786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf981396-93ea-40c2-8645-0802b0b07c47 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.556036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:47.558866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 354/713] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:47 2022] GET /identity/v3/role_assignments?user.id=62ef235951c84f62b91e30d56860a904&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:47.714548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d2c9b-7e25-412d-a7c2-a7478180f109 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:47.772460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e38f0-4616-4878-9521-33601c800acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:47.773069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38f0-4616-4878-9521-33601c800acf None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:47.773357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38f0-4616-4878-9521-33601c800acf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:47.773642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38f0-4616-4878-9521-33601c800acf None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:47.774411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e38f0-4616-4878-9521-33601c800acf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:47.775882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e38f0-4616-4878-9521-33601c800acf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:47.864644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 360/714] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:57:47 2022] POST /identity/v3/users => generated 254 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:48.375870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a687e8-71a6-44e7-9bcf-e183753cf3c1 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:48.376259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a687e8-71a6-44e7-9bcf-e183753cf3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:48.376700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a687e8-71a6-44e7-9bcf-e183753cf3c1 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:48.378468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 355/715] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:48.384142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:48.384438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:48.384722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:48.402575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:48.404340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:48.494876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 361/716] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:48.538321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a687e8-71a6-44e7-9bcf-e183753cf3c1 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:48.665402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a687e8-71a6-44e7-9bcf-e183753cf3c1 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:48.706239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:48.706239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:48.706239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:48.706239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] PATH_INFO: `/v3/services/network` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:48.706651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:48.706651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:48.720133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:48.721172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:48.722032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:48.728312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 356/717] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:48 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:48.730367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:48.890463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2c25c-daea-424a-9595-5e210b3389ee None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:48.936735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:48.937347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:48.937634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:48.937909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:48.938750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] RBAC: Adding query filter params (name=network) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:48.939062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:48.940455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54adbe1-c21d-48b5-bb66-4c4a5947f14e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:48.950309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 362/718] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:57:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:48.955930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:49.077300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f6bb5-9fff-4c61-bac2-797fc518a004 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:49.121432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:49.123271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.123793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.124246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:49.124999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] RBAC: Adding query filter params (type=network) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:49.125316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:49.127134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817477-d369-4997-87af-764fa1ab8630 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:49.138357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 357/719] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:57:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:49.227010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7f40-f8f1-490b-8381-de2b94806e96 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.227202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7f40-f8f1-490b-8381-de2b94806e96 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.227493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7f40-f8f1-490b-8381-de2b94806e96 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:49.229204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 363/720] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:49.235205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.235633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.236015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:49.255524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:49.256759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:49.331295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 358/721] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:49.367170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d7f40-f8f1-490b-8381-de2b94806e96 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:49.485872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d7f40-f8f1-490b-8381-de2b94806e96 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:49.524489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:49.525295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.525586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.525863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:49.537268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:49.545687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:49.546684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:49.547555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 364/722] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:49.556557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:49.714306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77feb-61d4-4164-9fef-ae59b03a654e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:49.770536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:49.771136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.771417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.771687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:49.772444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:49.772752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:49.774142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:49.786734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 359/723] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:49.792489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:49.931269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c039-0676-4fd6-ad66-2a60cfadc99a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:49.989094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:49.990019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:49.990337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:49.990617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.009170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.010724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-86c3fd53-2026-4544-a7dd-facff96e55d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:50.030156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:50.031118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:50.032057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:50.032057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:50.032057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:50.032057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:50.032762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 365/724] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:50.038455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:50.160117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019753-a9b9-49eb-9ca3-614d04c30969 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.160399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019753-a9b9-49eb-9ca3-614d04c30969 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.160678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019753-a9b9-49eb-9ca3-614d04c30969 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.162358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 366/725] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:50.167598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.167883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.168174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.169930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3b35-67f2-415c-a540-f797e39e6e2a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:50.201219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:50.202503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:50.213507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:50.214193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.214502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.214782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.215517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:50.215827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.217267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.239723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 360/726] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:50.246657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:50.329258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 367/727] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:50.358298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8164caf-7eb8-4848-acb9-fabf8ef49870 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:50.364471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:50.397350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:50.398473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.398901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.399317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.408377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.410235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:50.426942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:50.428175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:50.429345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:50.431624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 361/728] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:50.439078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:50.556759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd2e9b-5b5b-4524-b98d-3244f800c39a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:50.612305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:50.612908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.613187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.613457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.614330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:50.614639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.616163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.653123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 362/729] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:50.660305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:50.670570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6937fc3-fcdb-4fc1-991b-b0233f47e59b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:50.765610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:50.767734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.768056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.768450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.769580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.771241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bac803-f19a-4159-9889-91d6f7c6d84c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.851667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa3a-4615-4215-afd1-221c64566e18 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:50.878472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 368/730] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:57:50 2022] POST /identity/v3/services => generated 251 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:57:50.880634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de649c-c791-4de2-a5f1-9c94c00a010b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.885039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de649c-c791-4de2-a5f1-9c94c00a010b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.885039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de649c-c791-4de2-a5f1-9c94c00a010b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.885039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 369/731] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:50.893257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f30-e706-4701-8d44-474a189b98bf None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.893257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f30-e706-4701-8d44-474a189b98bf None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.893257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f30-e706-4701-8d44-474a189b98bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.920958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-93305f30-e706-4701-8d44-474a189b98bf None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:50.923159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-93305f30-e706-4701-8d44-474a189b98bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:50.925515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:50.926802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:50.927101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:50.927387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:50.948976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:50.952264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:50.981170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:50.983321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:50.984295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:50.984295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:50.984295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:50.984295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:50.985331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 363/732] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:50.990766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.080077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 370/733] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 10 13:57:51.131945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93305f30-e706-4701-8d44-474a189b98bf None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.157681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39fe9d-6e5a-4768-aa6c-b0897c552366 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.194503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.195567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.196062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.196549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.197498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:51.198005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:51.199663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed47ea0-8bc5-4173-ac6f-75a20cc95ff8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:51.218160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 364/734] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:51.218160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d50599-c2f2-4b94-b4ec-989a4438b14d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.218160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d50599-c2f2-4b94-b4ec-989a4438b14d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.218160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d50599-c2f2-4b94-b4ec-989a4438b14d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.218573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 365/735] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:51.222046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d50599-c2f2-4b94-b4ec-989a4438b14d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.293889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93305f30-e706-4701-8d44-474a189b98bf None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.335497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.336555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.337039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.337467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.352304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:51.356241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d50599-c2f2-4b94-b4ec-989a4438b14d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-3f037874-f87a-4988-a972-a16b203b349e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:51.361094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:51.362754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:51.363994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:51.371966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 371/736] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:51.374618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.375155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.375704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.397580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:51.399220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:51.498013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 372/737] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:51.499900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.560202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.561260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.561706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.562357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.572653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:51.574509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:51.589893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:51.591510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:51.592473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:51.594698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 366/738] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:51.595674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.636447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944c1c1-e046-4637-ab62-9c7fb8aeba4c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.674494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.675178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.675467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.675755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.676626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:51.676937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:51.678591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:51.692169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 373/739] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:51.692169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.710085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b106-84af-423a-9982-2843e49b57d6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.746753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.748743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.749177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.749620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.760845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:51.769367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:51.770500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:51.771511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:51.773933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 367/740] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:51.775069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.818020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6817-39b6-4321-a944-1e1f5f480f6a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.862657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.863749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.864242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.864747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.865788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:51.866414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:51.868145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:51.884164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 374/741] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:51.885463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:51.902582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37501583-fd81-4f89-a751-9771885a3c3d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:51.940998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:51.941896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:51.942176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:51.942472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:51.957238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:51.958992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:51.980652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:51.982160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:51.983147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:51.983147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:51.983147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:51.983147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:51.990680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 368/742] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:51.991874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.007108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8d8b4-da1c-451a-9b35-8281729446ee None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.054231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.054953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.055240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.055517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.056277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:52.056592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.057987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.067138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 375/743] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:52.067995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.102508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922aee2-7d45-408c-87ae-8c4e68893819 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.139325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.140238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.140495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.140770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.142816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=ffdd6ee2e1e64d2fb0d6d05687df82c1, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.144262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dddee6-c1c8-4a74-aaf4-bf48b055fb6a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.173923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b84a-35db-4770-a657-30e2df0a23b2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.208049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.208725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.209097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.209560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.210796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:52.211295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.213575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469233f4-d2f4-4e83-b7a7-3ffa4f3ee515 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 369/744] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:57:51 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058996-b492-48ca-bc0b-6f0cb157f060 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058996-b492-48ca-bc0b-6f0cb157f060 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62058996-b492-48ca-bc0b-6f0cb157f060 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 370/745] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:52.230467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62058996-b492-48ca-bc0b-6f0cb157f060 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.244347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 376/746] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:52.246857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.247911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.248402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.281293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:52.283709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:52.385175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62058996-b492-48ca-bc0b-6f0cb157f060 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.402656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 377/747] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 10 13:57:52.404239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.438055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.439244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.439703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.440160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.456051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.457800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:52.477654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:52.478612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:52.479513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:52.479513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:52.479513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:52.479513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:52.481133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 371/748] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:52.483116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.541420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a45df-64f7-4ef2-a9f3-81ead552ee83 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.587212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.588238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.588619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.588991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] PATH_INFO: `/v3/users/swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.600455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.601940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70293d-fe2b-4def-b502-3a899120b1ad None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.602253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:52.623455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:52.624640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:52.625606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 378/749] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:52.626691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.644729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.645761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.646272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.646741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.647892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.649448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.737925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 372/750] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:57:52 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:57:52.738986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.765485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb0558-b6c8-4cd7-afbf-483c1d4c28af None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:52.832634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:52.833719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:52.834277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:52.834845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:52.836254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:52.836882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:52.842517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:52.862861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 379/751] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:52.868512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:52.998073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181867-3884-4864-b32c-df9dc8b4eab7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.011780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341152-4e8c-4c77-ad5c-f0ef1d417259 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.048248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.049268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.049566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.049841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.050734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:53.051045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.055631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.058951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.061661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.062220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.062728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.083731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 373/752] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:53.091537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.217266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.218188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c4fbc-7470-4bae-aaeb-df6eaa374eb6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.218901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:53.235355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:53.236401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:53.237575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 380/753] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:57:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:53.242057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.303083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.303739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.303941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.304213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.350557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026ea7-fe5e-45ec-9bc4-3cc7dde269b7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.406479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.407986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.410610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.412241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.412997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.413716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.415168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:53.415820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.417529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.464900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 381/754] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:53.471565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.520608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 374/755] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:57:53.525942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.604686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ed900-afca-4bda-8673-6f847ae8cf2a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.641076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.642100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.642618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.643121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.663697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5a9d-4cee-4866-8457-175a2e0265bf None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.700345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.704735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.705032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.705294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.716873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.718670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adff755-74d1-4023-83a5-8f8893a500b3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.722231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.723763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-bfa96b21-4845-47d6-a0a7-a8f25d6d1f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:53.748311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:53.749990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 375/756] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:53.751784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbc18-cda9-447d-83c7-b7a60302a19d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.752361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbc18-cda9-447d-83c7-b7a60302a19d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.752361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fbc18-cda9-447d-83c7-b7a60302a19d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.753743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 376/757] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:53.759486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fbc18-cda9-447d-83c7-b7a60302a19d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.851595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 382/758] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:57:53.853030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.853319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.853628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.871292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:53.872746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:53.876949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fbc18-cda9-447d-83c7-b7a60302a19d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:53.914985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:53.915940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:53.916440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:53.916875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:53.917793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:53.918259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:53.921134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:53.933582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 377/759] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:53.935542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:53.950792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 383/760] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:57:53.951649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.046106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb2c8b-5e61-4977-8360-3cc40c7be853 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.081284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f348d-ef6e-4eaa-9c4d-17361d27ee4d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.083904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.084888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.085317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.085736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.098044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.099790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.117628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.118511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.118790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.119057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:54.119845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:54.120951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:54.123377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:54.123377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:54.123377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:54.123377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:54.125275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 378/761] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:54.126576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.131599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.133174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:54.148141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:54.149158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:54.150041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 384/762] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:54.150551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.241000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3e9a9-1ec7-4f34-80ec-561df26563cd None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.261406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e08a-8d1b-4b09-bed7-cb79afbcf74b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.279732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.280797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.281225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.281643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.282504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.284029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:54.297692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:54.298734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:54.299643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:54.300427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.300427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.300427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.300427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.300686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.300686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.301658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.303017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 379/763] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:54.304038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.322279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 385/764] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.323479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.436611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633f57a-12a3-472c-8bc0-cad54e4b5536 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.438780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5224f1c-8e92-4255-81b7-8cb5e79cac1e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.475066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.475568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.476267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.476553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.476729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.476825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.477324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.477553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.477865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.478119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.479149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.479334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.479820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.482873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.490460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 386/765] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.491542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.500521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 380/766] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.501986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.608197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767d37-4692-4e7f-a26d-c94d84fa6dd6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.619674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69b93-c833-4e00-ab6e-98d5fc51f299 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.662187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.663036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.663309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.663588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.667834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.669412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.669924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.670496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.671420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.671929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.673766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.674116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.675983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.680766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 381/767] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.681894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d70c2805-ea32-4b94-b5e5-2fdc6f14095b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:54.691886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:54.692793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:54.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 387/768] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:54.694601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b3e1-6849-45f0-b12a-fe3ff674e136 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.694702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b3e1-6849-45f0-b12a-fe3ff674e136 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.694967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b3e1-6849-45f0-b12a-fe3ff674e136 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.696363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 388/769] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:54.698236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b3e1-6849-45f0-b12a-fe3ff674e136 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.801984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d581c-d49f-41d5-9586-e9c5639c01f1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.823483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b3e1-6849-45f0-b12a-fe3ff674e136 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:54.853756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.854447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.854723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.855162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.855816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=62ef235951c84f62b91e30d56860a904) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.856676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.858262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.861650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:54.862453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.862616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.863285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.863688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] RBAC: Adding query filter params (interface=public, service_id=a9fb220d880f495f8735478eeb7dd1b6, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:54.864001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:54.865375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bbb75-ac85-4850-ac43-0999d6e494a0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:54.874574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 382/770] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/role_assignments?user.id=62ef235951c84f62b91e30d56860a904&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.874574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:54.882377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 389/771] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/endpoints?service_id=a9fb220d880f495f8735478eeb7dd1b6&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:54.883905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:54.884273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:54.884620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:54.909073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:54.910425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:55.011007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 390/772] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:57:55.012265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.037626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6258-647f-412b-983f-3765e5ec5a01 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.091259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.091895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.092001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.092242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.093045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:55.093358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.094850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:55.108064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 383/773] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:55.110242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.133855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89fc78-e320-4a7a-b85d-d39eeea204c0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.170915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.171602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.171883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.172162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.172862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.174305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:55.184022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 391/774] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:55.185697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.237221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba56ce7-b5f1-4c01-bad1-62881fae3e2f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.275978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125da75-d593-45df-973c-db4be8683130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.276701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da75-d593-45df-973c-db4be8683130 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.276986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da75-d593-45df-973c-db4be8683130 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.277264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da75-d593-45df-973c-db4be8683130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.306153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125da75-d593-45df-973c-db4be8683130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3125da75-d593-45df-973c-db4be8683130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:55.316711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:55.317763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:55.319606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 384/775] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:55.331242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125da75-d593-45df-973c-db4be8683130 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.357693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a870573-c3c9-4c8e-b560-328b5ab86e28 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.440149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.440838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.441117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.441410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.442271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=5dfa457ceb974f0b9c5200c213188dda) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:55.442585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.444217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c4888-1a82-47b7-a6ae-b0f39df918fc None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:55.451220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 392/776] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/role_assignments?user.id=5dfa457ceb974f0b9c5200c213188dda&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:55.531230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125da75-d593-45df-973c-db4be8683130 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.567813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082bdbe-202f-49fb-af9e-bf5a0e205542 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.568173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082bdbe-202f-49fb-af9e-bf5a0e205542 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.568376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082bdbe-202f-49fb-af9e-bf5a0e205542 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.570184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 393/777] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:55.572924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.573916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.574431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.575868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.576621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.576912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.577193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.577718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:55.578254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.579871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:55.590151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 385/778] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:55.598050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:55.598280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.599768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:55.738225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 394/779] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:57:55.752972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbb564-f090-44ee-b71f-bc383c8307e7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.759549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.799712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.800875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.801380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.801854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.822370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.825467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:55.854526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:55.855507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:55.857233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:55.857233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:55.857233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:55.857233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:55.859790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 386/780] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:55.867792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:55.923382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2401b-84ac-4a2d-b263-8b10d4861de6 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:55.970233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e33-8c73-4701-8005-667098710169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:55.971598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e33-8c73-4701-8005-667098710169 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:55.972368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e33-8c73-4701-8005-667098710169 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:55.972849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e33-8c73-4701-8005-667098710169 None admin] PATH_INFO: `/v3/services/network` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:55.973721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af9e33-8c73-4701-8005-667098710169 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:55.979670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af9e33-8c73-4701-8005-667098710169 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-94af9e33-8c73-4701-8005-667098710169 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:56.005939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:56.007224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:57:56.008060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:56.015675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 395/781] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:56.022423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e33-8c73-4701-8005-667098710169 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.037148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f1ba-c3d9-4e93-a652-449ea6d3911f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.073783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.074428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.074739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.075015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.075794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:56.076102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.077670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.091384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 387/782] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:56.107358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.210511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e33-8c73-4701-8005-667098710169 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.213138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595aa8-eb99-4297-89c0-a11a616c4ff7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.258917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.259723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.260004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.260279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.268082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.269837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.276124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.276812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.277099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.277379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.278242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:56.279216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.280082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.285434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 396/783] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:56.294233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:56.299537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:56.300487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:57:56.301424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:56.307977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 388/784] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:56.309667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.397107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610976cf-abd1-44fc-9144-9d335d59f0c3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.432381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.433119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.433406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.433976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.434534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] RBAC: Adding query filter params (type=network) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:56.434847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.436417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.437896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca0af-4047-422f-9b7a-a8a853abbeb4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.441976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 397/785] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:56.447811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.506929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.508000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.508522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.508998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.510054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:56.510689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.513051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.566739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 389/786] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:56.573470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.598393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fbdc9-34f1-4643-8442-78e7576e4a32 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.642427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] RBAC: Adding query filter params (interface=public, service_id=cc906d0e37d94c888f02fba29711838d, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.646221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.652442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 398/787] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/endpoints?service_id=cc906d0e37d94c888f02fba29711838d&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:56.658125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:56.769239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3f4-7d7d-4d75-b118-88e7d50bfd3a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:56.807096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:56.807776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:56.808066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:56.808427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:56.809064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:56.810496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:56.822246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 399/788] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:57:56.952528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25558f91-7db1-433c-b717-a0c61c3a90b7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:57.053367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:57.055239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:57.055239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:57.055239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:57.168886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:57.172677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:57.376793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 390/789] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:57:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 804 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:57:57.382346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:57.526252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d5735-3e42-4230-8f7e-c1fdaf7fe65d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:57.569915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:57.570754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:57.571044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:57.571320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:57.585451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:57.587000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-260f3597-9ce3-4532-8ca1-b44410ca91cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:57.613744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:57.614744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:57.616280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:57.616280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:57.616280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:57.616280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:57.616280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 400/790] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:57.620694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:57.748230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539ad4-facd-43b4-9d1a-778ae3f257b2 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:57.748633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539ad4-facd-43b4-9d1a-778ae3f257b2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:57.748867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539ad4-facd-43b4-9d1a-778ae3f257b2 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:57.750271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 401/791] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:57.755930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:57.756301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:57.756594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:57.775111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c82051-6143-4e78-9d65-d960870e31bc None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:57.790413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:57.791637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:57.816745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:57.818631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:57.819342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:57.821084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:57.822045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:57.822616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:57.825559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:57.849252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 391/792] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:57.854820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:57.920994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 402/793] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:57:57.942833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.049427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbc6f-0318-4e96-bfa9-b2a5c356060b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.085210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.086414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.086902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.087403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.094832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe1059-3d56-4cac-95a0-0a30db0db815 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.103691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.105231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:57:58.131168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:58.132149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:57:58.133051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:57:58.134698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 392/794] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:58.143167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.144419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.144918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.145397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.147278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.159026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:58.167175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:58.168727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:57:58.169684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:58.172485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 403/795] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:57:58.178513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.285472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e9041-ff75-431a-9e47-634555466074 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.321345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.321937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.322240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.322524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.323322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:58.323631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.325640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:58.338455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 393/796] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:58.347993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.383670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261462-f212-4128-9cc6-dd402fe6db47 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.423011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.424098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.424634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.425110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.426109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:58.426643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.428254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90c4b-495c-4a77-85e3-d4a9daf68b8a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:58.439201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 404/797] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:58.445217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c3977-225e-4fbf-b5a4-82be1df7f7ba None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.446166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c3977-225e-4fbf-b5a4-82be1df7f7ba None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.446692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c3977-225e-4fbf-b5a4-82be1df7f7ba None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.448692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 405/798] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:58.449817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c3977-225e-4fbf-b5a4-82be1df7f7ba None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.478241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544a6c-28f0-4a64-a4e4-7f7dc7524a77 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.508249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.508951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.509285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.509569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.510443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:58.510761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.512991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d1425-0f67-461a-93bf-abf86746ea7a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:58.519889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 394/799] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:58.521346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b261a-0a1b-4b1f-872b-0a8f58eaa49a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.521643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b261a-0a1b-4b1f-872b-0a8f58eaa49a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.521923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b261a-0a1b-4b1f-872b-0a8f58eaa49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.540467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-650b261a-0a1b-4b1f-872b-0a8f58eaa49a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:58.541718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-650b261a-0a1b-4b1f-872b-0a8f58eaa49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:58.569746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c3977-225e-4fbf-b5a4-82be1df7f7ba None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.610277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.611026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.611315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.611614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.629943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 395/800] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:57:58.630373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.631374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e41a-7a7a-45c9-a1d3-27d3afe881db None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.631665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e41a-7a7a-45c9-a1d3-27d3afe881db None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.631947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92e41a-7a7a-45c9-a1d3-27d3afe881db None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.632085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:58.633651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 396/801] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:58.640019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.640329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.641304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.668936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:57:58.670164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:58.670328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:58.673728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:58.676734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:58.676734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:58.676734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:58.676734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:58.676734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 406/802] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:58.707102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.779013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 397/803] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:57:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:57:58.783230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.837126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f5aea-aa7b-44e5-a6bd-6177dc1cbe05 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.879534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.880346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.880631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.880906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.892774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:57:58.901150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:58.902370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 407/804] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:58.903737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:58.925251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207b598-bbfa-4a41-8a0e-c75e3e9c161d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:58.976438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:58.977612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:58.978433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:58.979165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:58.980236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:58.980716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:58.983104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.000206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 398/805] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.001883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.041122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dcb5f-1d26-49b8-8919-963bc4f15ef4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.077243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.078066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.078437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.078718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.079384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.080776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:59.094536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:59.095578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.096461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 408/806] 10.222.0.58 () {56 vars in 1129 bytes} [Tue May 10 13:57:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:59.097328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.140086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2768f6-5bf0-447d-b0f1-7d4f952c63c5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.186241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.187229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.187716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.188191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.189129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:59.189625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.191209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.200827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 399/807] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.201976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.223090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53057a2-32cf-46dd-aaeb-70794b4091d7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.264463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.266290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.266806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.267550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.282284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.284640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:57:59.310006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:59.311069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:57:59.312127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.315160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 409/808] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:59.316182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.327343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d08fb-0cf9-4ee9-ae6c-8483847be592 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.364946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.365937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.366451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.366956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.367858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:59.368352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.369970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.376236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 400/809] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.377409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.426614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868fbb9-25ee-4b9c-8bf8-9c2963b18905 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.461664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.462592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.462946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.463316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.476065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.477522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.487528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04e452-96ab-4ca3-94fa-8f92e99c016f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:59.497348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:59.498371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:59.499281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:59.499281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:59.499281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:59.499281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.504065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 410/810] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:59.505133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.529686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.530730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.531233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.531842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.532921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:59.533436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.535089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.554151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 401/811] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.555456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.622352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cd885-5626-4ddc-9a6e-db3c537baaaf None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.663214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.663979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.664328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.664670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.665679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:59.666370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.668219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.678589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e38e8-b932-4754-b7fa-a58ddd8724b0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.678752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 411/812] 10.222.0.58 () {56 vars in 1124 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.678752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.711068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.712179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.712179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.712371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.713036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:57:59.713346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.714870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b23d2-9822-4be7-bd6b-6892536cfdc3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.728928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 402/813] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:57:59.730340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee2be-2d73-43ea-831a-be06742201e8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.730661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee2be-2d73-43ea-831a-be06742201e8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.730942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee2be-2d73-43ea-831a-be06742201e8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.732553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 403/814] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:57:59.733083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecee2be-2d73-43ea-831a-be06742201e8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.788206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a3e0b-99a1-49f4-a372-b77bbde1c3d6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.822119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.823022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.823307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.823585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.836925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.838411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.840703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecee2be-2d73-43ea-831a-be06742201e8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:57:59.855453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:57:59.856366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:57:59.857279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:57:59.857279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:57:59.857279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:57:59.857279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:57:59.857279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 412/815] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:57:59.857566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:57:59.881848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:57:59.882579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:57:59.882858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:57:59.883126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:57:59.883904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:57:59.885663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:57:59.967334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f59bf-2394-44ba-8158-824e46362e52 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.007478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.008309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.008593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.008867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.017666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.021542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-26037f94-b2ec-4c74-9010-96761edcef03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:00.034386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:00.035433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:00.036376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 413/816] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:57:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:00.036978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.037380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.037380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.054452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:00.055666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:00.090296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 404/817] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:57:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:00.090296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.173746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 414/818] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:00.175552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.234285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fd1f6-ea6d-47e3-a035-da68d232dfa8 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.281624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.282344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.282631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.283269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.284016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:00.284681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.286023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.306298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 405/819] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:00.307272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.315006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a1007-8f1c-4777-b8a2-61503c40eb4c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.356687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.357376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.357662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.357941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.358834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:00.359146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.360594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.380250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 415/820] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:00.381248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.423035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477f734-765e-4506-8a36-98fed52eadba None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.464417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.466462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.466901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.467348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] PATH_INFO: `/v3/services/network` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.468405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.470675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.484866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a3f2-8249-48b8-926f-c04f3ac4793a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:00.491929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:00.494386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:00.496810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 406/821] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:00.498164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.528182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.528993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.529286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.529687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.537098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.538587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:00.553757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:00.555993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 416/822] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:00.557568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.625455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93605d-fed2-4c14-adfe-221849a48088 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.664682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dae3b-c680-47e8-8f7b-fd2cec11f5c6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.673624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.674831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.675118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.675395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.688871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.690481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.707162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.707864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.708373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.708530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.709237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:00.709547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.715568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:00.721793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:00.722679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:00.724092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:00.724092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:00.724092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:00.724092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:00.729486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 407/824] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:00.729486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.729730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 417/824] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:00.731008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.837457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3ce64-db47-4c7d-8bc8-0d4812383556 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.840532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96b551-1bf7-4c25-91ed-1a45d6f7aad1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:00.876556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.877308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.877591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.877906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.878782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:00.879082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.880569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.882570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:00.883578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:00.884077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:00.884597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:00.885579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:00.886112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:00.888111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:00.898910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 408/825] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:00.899921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:00.903396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 418/826] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:00.904506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.010028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacee38-07e1-4ab9-80c1-06c9be3d32e1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.013087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16743e9-4d39-483d-b34b-5bb3eb23c36f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.049486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.049889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.050234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.050587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.050873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.051592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:01.051901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.052017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.052503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.052991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/62ef235951c84f62b91e30d56860a904/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.053278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.055573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=62ef235951c84f62b91e30d56860a904, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.057456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd3e5f-2beb-424b-afe5-1245daf57ddd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.059881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 409/827] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:01.060484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 419/828] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:00 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/62ef235951c84f62b91e30d56860a904/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6e41-8696-4a86-9479-dd7f049ef161 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6e41-8696-4a86-9479-dd7f049ef161 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6e41-8696-4a86-9479-dd7f049ef161 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 420/829] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:01.150235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6e41-8696-4a86-9479-dd7f049ef161 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.188500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225014-e8c6-41e2-9016-e79c969522e9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.248177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.249400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.249940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.250677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.259357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.261600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-c49c4bd2-ef1f-4703-ae8e-8e079202c059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:01.278759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:01.279721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:01.280605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:01.286224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 410/830] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:01.286224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.286224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.286224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.287159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6e41-8696-4a86-9479-dd7f049ef161 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.307436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:01.309268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:01.329746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.330526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.330821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.331098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.331874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.333295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.398238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 411/831] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:01.398238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.402232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 421/832] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:01.426763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.519854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1b38d-f013-469d-a655-5c6c6903df82 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.564493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.565180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.565459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.565848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.566677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:01.566894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756024-af8e-4446-8c78-5199b746b58e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.567339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.568597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.581047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 412/833] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:01.598757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.623829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.624691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.624981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.625523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.626416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.627924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:01.641822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:01.644173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:01.646280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:01.648265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 422/834] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 10 13:58:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:01.651525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.762004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500b40-4959-4883-a4bb-e039fc853fc3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.767915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273d41c-89db-4719-91a4-bc922dd645f5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.800068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.801116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.801396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.801667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/5dfa457ceb974f0b9c5200c213188dda/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.803639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.804299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.804581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.804856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:01.805579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:01.805886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.807375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.817592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 423/835] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:01.818937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:01.830230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=5dfa457ceb974f0b9c5200c213188dda, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:01.830230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72424a-e46d-4f1f-b0b9-fd0d633d19c6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:01.924698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 413/836] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:01 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/5dfa457ceb974f0b9c5200c213188dda/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 13:58:01.953301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1246-fa16-40ef-86fd-5c13e9b37e7c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:01.997292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:01.998419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:01.998926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:01.999430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:02.000724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:02.001263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:02.003017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f19dfd-c39b-4735-b679-26b89c7aa3fc None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:02.009071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 424/837] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:02.619411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8e97-6c5e-46f1-a09e-7cb5ba030c7b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:02.619963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8e97-6c5e-46f1-a09e-7cb5ba030c7b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:02.619963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8e97-6c5e-46f1-a09e-7cb5ba030c7b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:02.622297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 414/838] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:02.628037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:02.628475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:02.628668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:02.645549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:02.646747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:02.726298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 425/839] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:02.748442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8e97-6c5e-46f1-a09e-7cb5ba030c7b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:02.853760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8e97-6c5e-46f1-a09e-7cb5ba030c7b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:02.889805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:02.891028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:02.891544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:02.892022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:02.892871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:02.894479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:02.907512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:02.908569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:02.909408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:02.912236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 415/840] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:02.920833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:03.028393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2a2f-91bb-4957-9f62-9d4398a33be2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:03.065145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:03.065819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:03.066105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:03.066414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:03.067125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:03.067436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:03.068803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81401e-7b3e-4685-9d55-f6ecb679a17c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:03.073819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 426/841] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:03.083248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:03.189523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2b97a-6f56-422f-9a5f-a158270d8b1d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:03.227041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:03.228095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:03.228607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:03.229084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:03.230727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:03.231418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:03.233057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff308-57af-44cf-96da-eee165d34fff None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:03.238175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 416/842] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:03.656846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77418f-b474-4429-93da-ea2f5981a468 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:03.657378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77418f-b474-4429-93da-ea2f5981a468 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:03.657378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77418f-b474-4429-93da-ea2f5981a468 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:03.658947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 427/843] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:03.664608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:03.668845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:03.668845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:03.684432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:03.685940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:03.761663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 417/844] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:03.784866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77418f-b474-4429-93da-ea2f5981a468 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:03.892427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77418f-b474-4429-93da-ea2f5981a468 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:03.929274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:03.929995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:03.930307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:03.930585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:03.942299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:03.951689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:03.952710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:03.954018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 428/845] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:03.959433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.089293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7a8dd-de7a-4d60-b663-1b77ffa7222e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.126907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.127604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.127886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.128159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.128931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:04.129240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.130717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2b0b1-84be-49f0-8d02-1089d13b5428 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.142286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 418/846] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:04.147534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.263985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2010-960c-4aba-ad09-5caa568555cb None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.300143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.300952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.301244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.301520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.313258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.314840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.327521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938401c4-79f3-4548-9450-20bd14b03ed0 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.328160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938401c4-79f3-4548-9450-20bd14b03ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.328637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938401c4-79f3-4548-9450-20bd14b03ed0 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.330568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 419/847] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:04.333809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:04.334873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:04.335833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:04.335833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:04.335833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:04.335833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:04.335833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 429/848] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:04.338744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b68b-d9e0-4fcf-b112-cdd2468e07d9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.339819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b68b-d9e0-4fcf-b112-cdd2468e07d9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.340191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b68b-d9e0-4fcf-b112-cdd2468e07d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.342599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.362130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-21e6b68b-d9e0-4fcf-b112-cdd2468e07d9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:04.363554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-21e6b68b-d9e0-4fcf-b112-cdd2468e07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:04.454256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 420/849] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:04.455986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7045-4377-4ab5-9d75-222313d32b08 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.456449 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7045-4377-4ab5-9d75-222313d32b08 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.457047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7045-4377-4ab5-9d75-222313d32b08 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.459486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 421/850] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:04.466039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.466406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.466600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.473166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff836e08-4dd8-4305-b49d-b3168f921db4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.491221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:04.492422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:04.509415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.510529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.511571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.512779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.513790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:04.514388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.516053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.532975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 430/851] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:04.534103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.563232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 422/852] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:04.563892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.648680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172d41e-2b11-4aad-ba6f-d7acf9129b28 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.671673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220fa2-0a99-475b-9fc9-e4dbf0a7dfff None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.689173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.689835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.690120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.690468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.704803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:04.712086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:04.713359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 13:58:04.714316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:04.720185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 431/853] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:04.720185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.727891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.728713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.728997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.729273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.740841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.743383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:04.762312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:04.763418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 423/854] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:04.769356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.852330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cae07b-a7e9-4c33-9188-a6be91c97f4d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.868769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb07ff-1e29-4925-96ae-0751a9bf8367 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:04.893670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.895735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.895735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.895735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.896021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.897293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.913669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:04.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:04.927358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 424/855] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:04.927964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:04.966259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 432/856] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:04 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:04.966259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.056344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564db89-7c7b-4195-9334-23736f67708d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.116600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.117481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.118109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.118492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.119584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:05.119998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.120337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657f85-dd4d-4f62-af65-c1c7c939014f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.121742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.150254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 425/857] 10.222.0.58 () {56 vars in 1135 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:05.153672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.175181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.176164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.176452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.176731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.194063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.195143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:05.216876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:05.217837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:05.218789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:05.218789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:05.218789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:05.218789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:05.218789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 433/858] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:05.224636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.292952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6b98-04db-4a38-815b-891547f14841 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.331843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.332424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.332705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.332978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.358512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c56cc-f3af-4008-b402-fdcfdebb97e8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.395869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109332-076e-453d-9146-5d454ae6016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.396983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109332-076e-453d-9146-5d454ae6016f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.397481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109332-076e-453d-9146-5d454ae6016f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.397953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109332-076e-453d-9146-5d454ae6016f None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.398943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109332-076e-453d-9146-5d454ae6016f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:05.399463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109332-076e-453d-9146-5d454ae6016f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.401109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109332-076e-453d-9146-5d454ae6016f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.406915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.408391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.416590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 434/859] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:05.424128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109332-076e-453d-9146-5d454ae6016f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.520565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 426/860] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:58:05.521707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.539406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109332-076e-453d-9146-5d454ae6016f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.579280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.580138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.580542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.580846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.591363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.592771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:05.610914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:05.611909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:05.612848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:05.619924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 435/861] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:05.622358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.630641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985627-5e84-45d8-ba94-8ce30e2ff2b9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.671627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.674981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.674981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.674981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.685136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.686639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-963f00ff-ca66-42bb-b147-db176575c316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:05.705592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:05.711654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 427/862] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcae427-079d-4fa3-b258-414754ad5803 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcae427-079d-4fa3-b258-414754ad5803 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.712678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcae427-079d-4fa3-b258-414754ad5803 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.714904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 428/863] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:05.715789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcae427-079d-4fa3-b258-414754ad5803 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.744795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20538a58-42a8-42d8-8e47-0abec1b58756 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.783574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.784250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.784536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.784816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.785717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:05.786035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.787557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993565-16a2-43de-b9c3-9634565c1285 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.808156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 436/864] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:05.810151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.810531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.810870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.828563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:05.829907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:05.839062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcae427-079d-4fa3-b258-414754ad5803 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:05.873962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:05.874596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:05.875143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:05.875418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:05.876145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:05.876464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:05.877840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:05.895880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 429/865] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:05.895880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:05.918930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 437/866] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:05 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:05.931167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.011648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709e2a9-b966-4adf-aa3c-c244e5b21516 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.044633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabb26f-1074-4072-b7c4-c7b6a89ea271 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.049810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.050518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.050799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.051069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.092494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.093302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.093592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.093864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.102986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.104690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:06.123586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:06.125518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:06.127068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 438/867] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:06.130607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.141503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.142988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.245201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e550ce-66fd-4276-a30b-6399ec459dfc None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.256742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 430/868] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:05 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:06.256742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.290973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.291653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.292017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.292320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.293036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.294471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.370642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1219d2-21a7-495e-a761-852a98df3cef None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.405180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 439/869] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:06 2022] POST /identity/v3/services => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:06.415312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.416442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.417869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.418406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.418790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.419639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:06.419951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.421520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.441414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 431/870] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:06.442806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.554364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6a84-1127-42cb-a141-2434a0e76adb None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.569599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896dbba-55b6-473c-821b-bf13052db218 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.592569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.593566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.593683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.593921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.605586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.608162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.618758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.619780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.620813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.621311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.622249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=62ef235951c84f62b91e30d56860a904) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:06.622575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.624036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:06.631649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:06.632600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:06.633487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:06.633487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:06.633487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:06.633487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:06.633487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 440/872] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:06.638290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 432/872] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:06 2022] GET /identity/v3/role_assignments?user.id=62ef235951c84f62b91e30d56860a904&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:06.642701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.768346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0c1-4f92-43f6-b8ce-1c4645da8f78 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.807322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.808073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.808359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.808637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:06.809391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:06.809699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:06.811157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:06.826324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 441/873] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:06.831959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:06.951855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51efa-8eab-4c46-a5e6-94b40e2f40f9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:06.990869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:06.991595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:06.991880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:06.992150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.001477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.003757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:07.018956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:07.019923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:07.020922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 433/874] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:07.025762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:07.141227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786221-54f3-4a17-9b41-05fe668dd3a4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:07.179531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:07.180223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.180512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.180792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.181611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:07.181925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.184043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a9f85-ee8a-4bfa-8f3f-1eaffc0549af None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.197737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 442/875] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:07.203770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:07.310547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d71be-efa7-4a22-94f6-9d13c8e1ab90 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:07.348163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:07.348826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.349111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.349391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.350255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=5dfa457ceb974f0b9c5200c213188dda) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:07.350574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.351999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d3a5-0fdf-47d0-9301-bcd3eb8258de None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.359005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 434/876] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:07 2022] GET /identity/v3/role_assignments?user.id=5dfa457ceb974f0b9c5200c213188dda&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:07.369637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc977e62-2af3-4ae9-abbd-701f0c07d27f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.370537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc977e62-2af3-4ae9-abbd-701f0c07d27f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.370774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc977e62-2af3-4ae9-abbd-701f0c07d27f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.372401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 443/877] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:07.379105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.379453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.379658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.398414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:07.399638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:07.478847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 435/878] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:07.502501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc977e62-2af3-4ae9-abbd-701f0c07d27f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:07.616360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc977e62-2af3-4ae9-abbd-701f0c07d27f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:07.653959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:07.654903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.655212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.655487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.656152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.657544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:07.672785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:07.675585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:07.676430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 444/879] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 10 13:58:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:07.681925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:07.793579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f3851-22ea-4899-9232-6f9d1b0eacdb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:07.828664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:07.829344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.829626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.829904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.830658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:07.830967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.832337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.837870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 436/880] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:07.844494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:07.953939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615896be-86f6-438d-bb73-0935fa2e673c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:07.989594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:07.990306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:07.990594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:07.990872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:07.991594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:07.991907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:07.993423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:07.999190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 445/881] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:08.005123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:08.108664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd9e3f-1fff-4fdc-a531-5c3ec9877e01 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:08.144843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:08.145514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:08.145801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:08.146084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:08.146913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] RBAC: Adding query filter params (interface=public, service_id=59b215e8202f43928bcbadfafe32350b, region_id=RegionOne) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:08.147287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:08.148803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd13f08-d51b-4aff-bd41-bc8110057692 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:08.157106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 437/882] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:58:08 2022] GET /identity/v3/endpoints?service_id=59b215e8202f43928bcbadfafe32350b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:08.162559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:08.268389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee08b-e368-430b-879c-a886b2bcdc80 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:08.303313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:08.304006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:08.304292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:08.304572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:08.305271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:08.306815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0661d-3880-4ae5-ae14-95ec4c526fda None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:08.313423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 446/883] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:08 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:08.867115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496672c-1126-41e4-98e4-2bb33667d94a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:08.867665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496672c-1126-41e4-98e4-2bb33667d94a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:08.867665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496672c-1126-41e4-98e4-2bb33667d94a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:08.869299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 438/884] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:08.876557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e090b7-64d0-45fb-92ba-3865ffee114a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:08.877031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e090b7-64d0-45fb-92ba-3865ffee114a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:08.877107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e090b7-64d0-45fb-92ba-3865ffee114a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:08.894647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f0e090b7-64d0-45fb-92ba-3865ffee114a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:08.895840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f0e090b7-64d0-45fb-92ba-3865ffee114a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:08.975408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 447/885] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:08.997837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496672c-1126-41e4-98e4-2bb33667d94a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.122857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496672c-1126-41e4-98e4-2bb33667d94a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.127876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a534-c0f1-44e7-b06f-59794f6fe8ff None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.128200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a534-c0f1-44e7-b06f-59794f6fe8ff None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.128437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a534-c0f1-44e7-b06f-59794f6fe8ff None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.130098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 448/886] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:09.136425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.137098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.137722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.159128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:09.160327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:09.161055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.161903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.162189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.162493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.163151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.165128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:09.177868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:09.179077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:09.179980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:09.184862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 439/887] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:09.186124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.253927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 449/888] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:09.275299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.292754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6dcec-6a57-4a79-b0ee-5c1b220f79ec None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.335165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.336212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.336724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.337222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.338133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:09.339261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.340933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.347634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 440/889] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:09.353682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.388916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc870-0060-4b29-9f4c-bcfb36652318 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.425581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.426473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.426757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.427032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.427692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.429170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:09.441964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:09.443047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:09.444130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 450/890] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:09.449468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.484206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1c01-65c3-4417-99ed-3b330ef36f0c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.519353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.520291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.520728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.521150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.522027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:09.522555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.524127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.535622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 441/891] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:09.537305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.554720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1da6-acf5-43e5-b04c-333a871c2894 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.590724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.591578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.591866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.592142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.592851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:09.593160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.594552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.605630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 451/892] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:09.606806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.643312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c37d5-1923-4c0d-a6a2-8d09ca2ea3d6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.681419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.682559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.683336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.683656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.684624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] RBAC: Adding query filter params (interface=public, service_id=8e3696fb8610448ab9e634726939d5be, region_id=RegionOne) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:09.685109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.686794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.697437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 442/893] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/endpoints?service_id=8e3696fb8610448ab9e634726939d5be&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:09.699794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:09.748277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78778fac-dcde-4dcc-afa9-658806bce248 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.783828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.784429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.784838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.784957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.785667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:09.785970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.787370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32af4e8-c49d-4553-9684-bb3bda9f0644 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.798611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 452/894] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:09.809611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b880-1be7-4f24-b891-e3396dc19a71 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:09.849340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:09.850417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.850850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.851272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.852123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:09.853737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858239e-8646-4fc2-ad16-4d8672b5be0d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:09.866353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 443/895] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:09 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:09.877315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00227a8-ccd1-4003-8037-c634f70e86dc None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.877863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00227a8-ccd1-4003-8037-c634f70e86dc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.878115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00227a8-ccd1-4003-8037-c634f70e86dc None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.880294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 453/896] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:09.887977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:09.888554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:09.888554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:09.912685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:09.913916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:09.996184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 444/897] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:10.021555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00227a8-ccd1-4003-8037-c634f70e86dc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:10.144379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00227a8-ccd1-4003-8037-c634f70e86dc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:10.180497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:10.181322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.181667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.181943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.193926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:10.202453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:10.203410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:10.204256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 454/898] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:10.211655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:10.331031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439678ee-7525-40aa-ae51-dfee368208e2 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:10.366539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:10.367225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.367508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.367786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.368536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:10.368845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:10.370817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:10.380573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 445/899] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:10.387645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:10.500536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b430709-bfae-4ba5-b307-25b1c505881e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:10.536192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:10.536992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.537277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.537839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.548986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:10.550474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:10.570664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:10.571733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:10.573172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:10.573172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:10.573172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:10.573172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:10.573172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 455/900] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:10.579837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:10.686059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9df7e-cf3f-4758-9c21-2eb4bf6533b6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:10.725007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:10.725929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.726461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.726977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.727909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:10.728440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:10.730840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613438b-e302-4146-8fc7-9921edde71fa None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:10.743624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 446/901] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:10.750308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:10.860722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86893f80-dbd3-4c7b-b4c4-a3f888f8a347 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:10.896191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad641c2a-2dd5-4331-8e37-c339fb6352d8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.896745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad641c2a-2dd5-4331-8e37-c339fb6352d8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.896745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad641c2a-2dd5-4331-8e37-c339fb6352d8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.898270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 447/902] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:10.902303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82815-ba50-4097-887c-79b956841ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:10.903214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82815-ba50-4097-887c-79b956841ded None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.903547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82815-ba50-4097-887c-79b956841ded None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.904032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82815-ba50-4097-887c-79b956841ded None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.908594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:10.909129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:10.909607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:10.911702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82815-ba50-4097-887c-79b956841ded None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:10.913156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82815-ba50-4097-887c-79b956841ded None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:10.930154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-92a82815-ba50-4097-887c-79b956841ded None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:10.930835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:10.931756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:10.933227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 456/903] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:10.938697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:10.940122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82815-ba50-4097-887c-79b956841ded None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.011947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 448/904] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:11.034867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.053703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82815-ba50-4097-887c-79b956841ded None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.098754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.099825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.100326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.100982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.102545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:11.103200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.105036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.133255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 457/905] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:58:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:11.136302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.149952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba8d47-5afc-4834-8509-1c3569cfa76e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.196112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.198676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.199183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.199657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.200990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.203078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:11.218258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:11.219429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:11.220282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 449/906] 10.222.0.58 () {56 vars in 1132 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:11.226785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.247663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b136b9b-c57e-4f04-9825-dd9b401e38e6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.289752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.291035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.292402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.292890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.339371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c0fd-602d-4512-b696-0ba75e134a14 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.364531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.366298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.375980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.376657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.376941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.377219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.377930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:11.378262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.379628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.389791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 450/907] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:11.390638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.476205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 458/908] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:11.481520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.512327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500d04-b49b-48a4-b784-4bf309529312 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.549190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.550546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.551038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.551523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.552457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:11.552965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.554580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.565332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 451/909] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:11.567176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.595774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca354-cdbd-4ed3-a8f4-2cf2bc7def5c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.630256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.631064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.631345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.631626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.643894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.646625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:11.682568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:11.683578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:11.684556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:11.684556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:11.684556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:11.684556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:11.685597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 459/910] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:11.693507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:11.698829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b8-bade-4286-af4c-b4dc7257800b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.752578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.753401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.753780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.754659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.755791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.757737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.802490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a8091-4a25-47ca-a184-af4ff7817cf8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:11.842695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 452/911] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:11.852956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:11.853882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:11.854178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:11.854537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:11.855360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:11.855676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:11.857198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:11.872317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 460/912] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:11.878684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.001507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20cafb-10bf-4887-a5ce-9e0a6673a315 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.039117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.039933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.040216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.040531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.049124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.050650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:12.066286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:12.067196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:12.068068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 453/913] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:12.074941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.196020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7d43-098e-4390-b289-94c6d477225d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.230976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.231656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.231938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.232264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.233065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:12.233376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.234847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058d8d-4203-4758-8ff3-6bd411b0d2c2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.251003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 461/914] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:12.256994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.324953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b645-9218-4421-a2f6-692561e5cd3e None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.325277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b645-9218-4421-a2f6-692561e5cd3e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.325504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b645-9218-4421-a2f6-692561e5cd3e None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.327215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 462/915] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:12.332485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.332767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.333049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.349531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:12.350753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:12.362012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aabcf-be74-4557-b676-e71c0174a2a0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.396081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.396985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.397272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.397549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.398411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] RBAC: Adding query filter params (role.id=4cdecf0e40a34a37af9266d132ea8fd7, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=5dfa457ceb974f0b9c5200c213188dda) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:12.398725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.400147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bbfd1-3ba2-42ec-98f0-56edb8c06392 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.406747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 454/916] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:12 2022] GET /identity/v3/role_assignments?user.id=5dfa457ceb974f0b9c5200c213188dda&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=4cdecf0e40a34a37af9266d132ea8fd7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:12.408175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bb44-d8fa-4cbf-9b48-d14b303708cb None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.408492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bb44-d8fa-4cbf-9b48-d14b303708cb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.408738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714bb44-d8fa-4cbf-9b48-d14b303708cb None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.418797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 455/917] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:12.418797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.418797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.418797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.438232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 463/918] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:12.438435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:12.438992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:12.453715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.542760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 456/919] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:12.559769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.569520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabefe15-5662-42a2-9ad3-e97a4012ae69 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.612336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.613165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.613455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.613736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.614609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.616314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:12.630724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:12.631759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 10 13:58:12.632616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:12.635732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 464/920] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:12.642536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.677510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f599-71d5-4cf3-9329-133c0b7c3c2f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.715083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.716927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.717418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.717915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.718934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.720541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.746743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ae2-e693-4f63-8357-11e8cf68d420 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.781890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.782608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.782897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.783173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.783888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:12.784259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.785790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.792848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 465/921] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:12.798150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:12.834452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 457/922] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:12 2022] POST /identity/v3/services => generated 241 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:12.925268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ec0c-2d9c-483a-af8b-a74031a38300 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:12.977535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:12.978226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:12.978562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:12.979582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:12.980316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:12.980627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:12.982209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b953c-8b2d-473a-8265-33abda42598c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:12.988285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 466/923] 10.222.0.58 () {56 vars in 1114 bytes} [Tue May 10 13:58:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:12.993692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:13.111544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987098af-70c4-42b2-a112-6bfdc0f1a772 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:13.146537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:13.147627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:13.148062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:13.148500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:13.149410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:13.151492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13768ddd-a1fa-44a5-a95e-08f6f463f721 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:13.258156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 458/924] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:58:14.300953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8180c2-6809-4e46-97a7-4c1fbd9d201f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.301990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8180c2-6809-4e46-97a7-4c1fbd9d201f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.302507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8180c2-6809-4e46-97a7-4c1fbd9d201f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.304395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 467/925] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:14.310904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.311785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.312289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.329991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:14.331549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:14.426328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 459/926] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:14.460107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8180c2-6809-4e46-97a7-4c1fbd9d201f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:14.580227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8180c2-6809-4e46-97a7-4c1fbd9d201f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:14.617276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:14.618079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.618394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.618670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.619888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:14.621333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6a127509-172f-45c1-9877-12b0a310cddd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:14.633591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:14.635416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:14.636692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 468/927] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:14.644023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:14.695502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b10e-dbd5-4d21-a851-5cf28b81815c None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.695878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b10e-dbd5-4d21-a851-5cf28b81815c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.696064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b10e-dbd5-4d21-a851-5cf28b81815c None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.697928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 469/928] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:14.703138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.703428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.703713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.727875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:14.730327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:14.751665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de732ca8-c6d5-4c7d-96e7-c89dfd12efd6 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:14.794401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:14.795149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.795708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.796005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.796719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:14.797027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:14.798416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:14.806275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 460/929] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:14.811388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:14.826260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 470/930] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:14.857283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:14.925778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041afd59-ceb5-4ef1-a346-4c6a27b6a0b4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:14.969082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:14.969751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:14.970032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:14.970348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:14.971194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:14.971829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:14.973445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:14.980986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 461/931] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:14.981681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf471-0493-4c1d-9e76-6b588f316fd3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.025508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.026939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.027401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.027894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.040499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:15.050051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:15.051112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:15.051967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:15.054674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 471/932] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:15.062221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.178331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1713e7d-1c62-4f6e-83c5-d458c5449fe9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.218759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.219704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.220128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.220542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.221426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:15.221895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.223416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76aab-add7-4381-b77c-84a9c4cb5231 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.233171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 462/933] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:15.239117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.254091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02ee6-5820-471d-96ba-3109e87a4f27 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.254409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02ee6-5820-471d-96ba-3109e87a4f27 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.254692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02ee6-5820-471d-96ba-3109e87a4f27 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.256308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 463/934] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:15.261923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.262257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.262524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.280736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:15.281226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:15.352754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dfe-116e-4e88-93e3-bf10369dc677 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.358326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 464/935] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:15.373721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.390518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.391683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.392117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.392534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.406537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.408401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:15.427623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:15.428832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:15.429917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:15.429917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:15.429917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:15.429917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:15.432463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 472/936] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:15.438368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.480915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4024c8-bed4-47c3-bbad-a74fd6071cff None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.516366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.517176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.517476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.518005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.518422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.519803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:15.533275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:15.534345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:15.535201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:15.539639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 465/937] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:15.540514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.546918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4db2d-fdac-4b00-a288-786c740df226 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.591439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.592457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.592746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.593026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.593771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:15.594082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.596011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfacb1-ef7c-4c44-9afb-f65e70b62691 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.612211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 473/938] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:15.613115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a291-cb95-4bbf-b887-b982b687a41b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.614000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a291-cb95-4bbf-b887-b982b687a41b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.614000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a291-cb95-4bbf-b887-b982b687a41b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.614852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 474/939] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:15.620197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612a291-cb95-4bbf-b887-b982b687a41b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.646264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e206089-6043-4e3d-bd7f-c17e4892d065 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.680339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.680840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.681121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.681508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.682771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:15.683083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.684444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043181c6-8b4c-49ef-88ff-6fcd74734087 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.694023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 466/940] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:15.694023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.694023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.694023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.712334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:15.714275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:15.738437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612a291-cb95-4bbf-b887-b982b687a41b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.772741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.773568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.773851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.774128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.782289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.783764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.794289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 467/941] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:15.794289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:15.799556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:15.801023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:15.802491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:15.805629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 475/942] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:15.807164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.915615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64f8f2-fe86-4957-a2df-b2c3b261c38b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.916356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc9ee5-7f1b-42c1-b880-ae1a6bb5b5c7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:15.952093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.952505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:15.952747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.953037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.953198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:15.953352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.953455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:15.953698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:15.954174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:15.954431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:15.954529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.954798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:15.956152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.956310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:15.967385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 468/943] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:15.967385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:15.977628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 476/944] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:15.979110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.074002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793826-142f-4c44-8c23-c58091db1112 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.086457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6b1d0-5bd1-4d41-bc79-11791da82ac0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.108815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.109479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.109762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.110039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.110770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.112240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.113615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING keystone.server.flask.common [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 10 13:58:16.124920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.125413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.125688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.125970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.126848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] RBAC: Adding query filter params (interface=public, service_id=f6b9c3e760a8441bad386bc0261e44b6, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:16.127247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.128618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.135158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 477/945] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:58:15 2022] GET /identity/v3/endpoints?service_id=f6b9c3e760a8441bad386bc0261e44b6&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:16.136061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.162611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.sql.core [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 13:58:16.162611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 13:58:16.162611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [parameters: {'id': 'cd659cff177742b99f4acc9a50652a91', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 13:58:16.162611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80413) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd659cff177742b99f4acc9a50652a91.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd659cff177742b99f4acc9a50652a91. May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application conn.query(q) May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:58:16.179372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result.read() May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 10 13:58:16.180344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return project_ref.to_dict() May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application next(self.gen) May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application yield resource May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application next(self.gen) May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.session.rollback() May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.force_reraise() May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise self.value May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application session.commit() May 10 13:58:16.181906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._prepare_impl() May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.session.flush() May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._flush(objects) May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application compat.raise_( May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application flush_context.execute() May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 13:58:16.182915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rec.execute(self) May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application c = connection._execute_20( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 13:58:16.183819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application conn.query(q) May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result.read() May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:58:16.184617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [parameters: {'id': 'cd659cff177742b99f4acc9a50652a91', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.185748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:16.186758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.Conflict( May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd659cff177742b99f4acc9a50652a91. May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 469/946] 10.222.0.58 () {60 vars in 1140 bytes} [Tue May 10 13:58:15 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:16.188824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.254433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b685e-0209-44fa-8bbb-eae982630354 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.294777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.295587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.295873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.296150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.297254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cdf30-cb81-4e26-b20c-b57f3a4e9bd5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.307020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.308965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:16.327639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:16.329062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:16.330193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:16.330193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:16.330193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:16.330193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:16.332052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 478/947] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:16.337793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.341038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.341839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.342272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.342651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.343446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.345126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.354844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 470/948] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:16.355834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.454245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f6e76-2d95-4fdb-839e-7195b56c631f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.468040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67864-78ee-4c03-bc68-f156fdd69b22 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.493285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.495196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.495643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.496095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.504712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.506581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.506837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.507456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.507844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.508245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.509153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:16.509631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.511394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:16.523621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:16.524791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:16.526108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:16.528725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 479/949] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:16.530598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 471/950] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:16.531592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.537146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.664604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a007a-dc85-4041-8dcd-0b73b2cbe8c4 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.682818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608486-9a07-480f-8d46-eae7c34361d5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.710363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.711301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.711745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.712176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.713795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:16.714462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.716090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.727006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.727910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.728467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.728718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.736926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.739036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.742090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 472/951] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:16.760375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:16.761460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:16.762454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:16.764993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 480/952] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:16.770486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:16.890859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773145-585c-479c-8227-88e18ddaf49e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:16.930670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:16.932349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:16.932785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:16.933210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:16.934220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:16.934689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:16.936402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4081d-3c39-41f8-bb23-6a6d9a2a6d61 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:16.950074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 473/953] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:16.966963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:17.069821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afa084-ff2a-4511-a5d2-5e65ba618c5f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:17.109829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:17.110839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:17.111125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:17.111401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/5dfa457ceb974f0b9c5200c213188dda/roles/4cdecf0e40a34a37af9266d132ea8fd7` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:17.113775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=5dfa457ceb974f0b9c5200c213188dda, role_id=4cdecf0e40a34a37af9266d132ea8fd7)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:17.115275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99360554-f041-40dd-bfa5-2a1c4c4ef504 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:17.187150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 481/954] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:16 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/5dfa457ceb974f0b9c5200c213188dda/roles/4cdecf0e40a34a37af9266d132ea8fd7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:58:17.280782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ae61-2fa7-45d6-84a8-09755802ee9d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:17.281148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ae61-2fa7-45d6-84a8-09755802ee9d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:17.281332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ae61-2fa7-45d6-84a8-09755802ee9d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:17.283064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 474/955] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:17.288983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e12a82-b5ea-4822-9608-3080751515ac None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:17.289287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e12a82-b5ea-4822-9608-3080751515ac None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:17.289536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e12a82-b5ea-4822-9608-3080751515ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:17.308284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-64e12a82-b5ea-4822-9608-3080751515ac None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:17.309495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-64e12a82-b5ea-4822-9608-3080751515ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:17.386231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 482/956] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:17.408831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ae61-2fa7-45d6-84a8-09755802ee9d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:17.517252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ae61-2fa7-45d6-84a8-09755802ee9d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:17.551478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:17.552205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:17.552488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:17.552761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:17.553512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:17.554969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0893d2-c64a-485a-b0e2-8726fd9e4b04 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:17.633839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 475/957] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:17 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:18.699874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34daa4c-da5d-4e7d-a132-3585247b9946 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:18.700405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34daa4c-da5d-4e7d-a132-3585247b9946 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:18.700405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34daa4c-da5d-4e7d-a132-3585247b9946 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:18.702052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 483/958] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:18.707835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:18.708299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:18.708378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:18.730188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:18.731741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:18.819142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 476/959] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:18.841742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34daa4c-da5d-4e7d-a132-3585247b9946 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:18.959267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34daa4c-da5d-4e7d-a132-3585247b9946 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.004335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.005606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.006552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.006552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.007140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.008599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:19.022398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:19.036803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:19.038926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 484/960] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:19.040839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.150378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa2b17-5e7c-4ecc-a87e-d6082289ef7d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.186558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.187384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.187664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.187939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.188650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:19.188957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.190366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d93eb1-dcb9-42ce-bf30-c97011860741 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.195826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 477/961] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:19.201579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.210723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7973-a931-41f7-b718-54fa17508851 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.210990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7973-a931-41f7-b718-54fa17508851 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.211209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7973-a931-41f7-b718-54fa17508851 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.213406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 478/962] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:19.218630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.218920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.219198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.240566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:19.240566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:19.318710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 479/963] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:19.324820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53e42f-5261-400d-9409-9a715a12f8f0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.344051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.372547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.374035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.374892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.375642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.376719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:19.377558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.379683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.391657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 485/964] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:19.394386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.454823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f74ae6-a38f-4b90-909a-37123487e1e3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.490666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.491687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.492166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.492652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.493588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.495249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.511084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de4aff-ce31-4fe6-aeb7-e29b9b2a95a5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.556518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.sql.core [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 13:58:19.556518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 13:58:19.556518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [parameters: {'user_id': 'ce429eb6b0234dffa0f8b47a8d4b55c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 13:58:19.556518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80413) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application conn.query(q) May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result.read() May 10 13:58:19.564754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:58:19.565665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application next(self.gen) May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application yield resource May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application next(self.gen) May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.session.rollback() May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.force_reraise() May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise self.value May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application session.commit() May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 10 13:58:19.566758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._prepare_impl() May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.session.flush() May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._flush(objects) May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application compat.raise_( May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application flush_context.execute() May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rec.execute(self) May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 10 13:58:19.567671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application _emit_insert_statements( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result = connection._execute_20( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ret = self._execute_context( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 10 13:58:19.568556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self.dialect.do_execute( May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result = self._query(query) May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application conn.query(q) May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application result.read() May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 10 13:58:19.569549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application packet.raise_for_error() May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ce429eb6b0234dffa0f8b47a8d4b55c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:19.573748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 10 13:58:19.576380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 480/965] 10.222.0.58 () {60 vars in 1131 bytes} [Tue May 10 13:58:19 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:19.579620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.583683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.583683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.583683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.583683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.585626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.587059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a0979-6727-48bb-b8c5-1e044f489569 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.670874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 486/966] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:19.686311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7e750-e1ad-45a4-bbd7-21998bc02921 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.716177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801ab2e-acf0-4892-82ca-8a9c837923db None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.716715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801ab2e-acf0-4892-82ca-8a9c837923db None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.716715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801ab2e-acf0-4892-82ca-8a9c837923db None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.718508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 487/967] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:19.731344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.731344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.731344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.738649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.739468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.739749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.740077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.750331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.751209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:19.752068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.752824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:19.768553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:19.769590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:19.770969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 481/968] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:58:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:19.776614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.838331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 488/969] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:19.857628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:19.893429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821cc22-3c44-498b-9dbd-9e0ce8cfbeee None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.940804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:19.941635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:19.942012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:19.942493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:19.943400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:19.943805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:19.945876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:19.981554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859c8b-00ae-463a-add1-cc228dee1185 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:19.983661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 482/970] 10.222.0.58 () {56 vars in 1101 bytes} [Tue May 10 13:58:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:20.026719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.027809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.028243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.028885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.041154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-910053a6-7569-46e6-b7e3-006d095b1f2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:20.050065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:20.051287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:20.052607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:20.055315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 489/971] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:20.062736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.068111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0aca-75ed-4aa6-999e-8becd8e6aaa5 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.069756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0aca-75ed-4aa6-999e-8becd8e6aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.069756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0aca-75ed-4aa6-999e-8becd8e6aaa5 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.071166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 490/972] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:20.077538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.077951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.078310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.101978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:20.104059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:20.196184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bea91c-27bc-4b3c-a235-4cb65c292b4e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.199468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 491/973] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:20.224518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.243833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.244665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.245024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.245371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.246228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:20.246724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.248269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.265384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 483/974] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:20.266874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.368521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3cdc2-0fe4-4464-a275-2e7f7505b3f7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.405035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282f08d-c390-4743-b831-baa9b334b59a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.412670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.413505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.413782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.414129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.414733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.416698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:20.429283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:20.430625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:20.432727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:20.434566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 492/975] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:20.436608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.462083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.463842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.464157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.464440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.482909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.484289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:20.504264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:20.511626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:20.512788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:20.512788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:20.512788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:20.512788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:20.512788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 484/976] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:20.514907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.559383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd89c3-3587-4ef5-9f07-38f9d5a0edab None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.598379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.599247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.599428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.599619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.600327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:20.600633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.602146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.612427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 493/977] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:20.613680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.628064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7f33-4e8a-4c60-93f7-9ed037dd1422 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.665665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.666391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.666561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.666836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.667622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:20.667938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.669486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.683592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 485/978] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:20.689294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.751422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a40ff9-2f46-45eb-a2ca-53a0a70ea6a6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.788265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.788939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.789329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.789611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.790888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:20.791505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.792901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.798309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 494/979] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:20.804777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.808678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cee72-9fac-46ec-af8f-2f6385b64e9f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.846277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.847100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.847383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.847657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.855744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.857332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:20.873961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:20.875174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 10 13:58:20.876393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:20.881665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 486/980] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:20.882310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:20.921429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa8db-9847-49b5-a59d-313324056d6f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:20.958341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:20.959103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:20.959452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:20.959789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:20.960655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] RBAC: Adding query filter params (interface=public, service_id=72267f37e333409cbfcabb3130ed0bbc, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:20.961059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:20.963533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:20.969182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 495/981] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/endpoints?service_id=72267f37e333409cbfcabb3130ed0bbc&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:20.974661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:21.000426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258003c7-e833-417d-b0bf-a3f1dcba2828 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.038806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.039493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.039780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.040970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.041484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:21.041995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.043469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.067059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 487/982] 10.222.0.58 () {56 vars in 1149 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:21.072696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:21.101407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a873731-ced6-48fa-ba0b-c470948bdfbd None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.137587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.138152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.138464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.138743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.139433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.140810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.154645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 496/983] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:20 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:21.176715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b8a88-665f-4b3d-9bd0-c07bf9e9235e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.214387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.215063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.215347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.216012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.290587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.291952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.405797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 488/984] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:21 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:21.411668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:21.514417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1e29a-36a7-4a9a-9bd7-f2c8b65eb576 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.548831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.549635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.549922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.550216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.566904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.568449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:21.585968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:21.587518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:21.588788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:21.588788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:21.588788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:21.588788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:21.590283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 497/985] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:21.593566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:21.707217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb56a02-7081-4e50-8380-c68798805c3b None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.748167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.749099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.749383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.749653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.750426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:21.750734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.752109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.769360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 489/986] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:21.770665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:21.892821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe59f91-7b40-4f74-8622-7d789bfa8685 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:21.934446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:21.935653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:21.936141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:21.936588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:21.944042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:21.945729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:21.960830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:21.961743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:21.963541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:21.963979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 498/987] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:21.970038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:22.098225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb8655-62b1-463e-91cb-de576469c832 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:22.134556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:22.135563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.136023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.136499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.137456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:22.137939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:22.139593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223cfd4-9d00-47ee-9f38-456e5258b501 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:22.153804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 490/988] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:22.159827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:22.195945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7749-b209-4b9d-b036-a7f375d8a4b2 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.196315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7749-b209-4b9d-b036-a7f375d8a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.196566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7749-b209-4b9d-b036-a7f375d8a4b2 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.198128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 491/989] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:22.204166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.204455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.204728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.222278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:22.223722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:22.289368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fc528-1a8c-4610-8431-46e660f61cf5 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:22.316893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 492/990] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:22.327065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:22.327740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.328077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.328357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.329205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] RBAC: Adding query filter params (role.id=4cdecf0e40a34a37af9266d132ea8fd7, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=5dfa457ceb974f0b9c5200c213188dda) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:22.329520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:22.330995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:22.339584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 499/991] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:22 2022] GET /identity/v3/role_assignments?user.id=5dfa457ceb974f0b9c5200c213188dda&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=4cdecf0e40a34a37af9266d132ea8fd7 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:22.340312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:22.461509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba481-707b-4de8-ba88-9617f4d1bb6d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:22.507044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:22.507793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.508078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.508358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.509726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:22.511331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:22.526466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:22.527546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:22.528400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 493/992] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:22.534644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:22.675873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7acc-c67b-48c2-b703-3814ee4b6ab9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:22.722718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:22.723421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.723702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.723977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.724705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:22.725009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:22.726406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:22.738290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 500/993] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:22.740840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:22.901857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e75f-f0dc-4c61-96d9-0b593146ef5f None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:22.971142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:22.971928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:22.972111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:22.972371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:22.973084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:22.973386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:22.974779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:22.981153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 494/994] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:22.987777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:23.131260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa91e6d-3349-4731-9b9f-5bd5785eb37b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:23.177223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:23.179264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:23.179783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:23.180335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:23.182146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] RBAC: Adding query filter params (interface=admin, service_id=f6b9c3e760a8441bad386bc0261e44b6, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:23.182485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:23.183880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84659369-e983-41ad-8fd6-5b9a2bd78c49 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:23.196806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 501/995] 10.222.0.58 () {56 vars in 1249 bytes} [Tue May 10 13:58:22 2022] GET /identity/v3/endpoints?service_id=f6b9c3e760a8441bad386bc0261e44b6&interface=admin®ion_id=RegionOne => generated 193 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:23.198998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:23.304220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a68175-c583-4c5f-b87f-f529e4e49f7d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:23.337588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:23.338507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:23.338619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:23.338835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:23.339498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:23.340872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58920aff-a98d-4976-a56f-de245a614263 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:23.347775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 495/996] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:23 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:23.623913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6169a-735a-47a3-8d49-78697397c304 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:23.624265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6169a-735a-47a3-8d49-78697397c304 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:23.624616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6169a-735a-47a3-8d49-78697397c304 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:23.626038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 502/997] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:23.631808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de0b9-093b-485a-9150-9423411a3273 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:23.632081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de0b9-093b-485a-9150-9423411a3273 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:23.632357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de0b9-093b-485a-9150-9423411a3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:23.659891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-944de0b9-093b-485a-9150-9423411a3273 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:23.661104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-944de0b9-093b-485a-9150-9423411a3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:23.750251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 496/998] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:23.770878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6169a-735a-47a3-8d49-78697397c304 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:23.875286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6169a-735a-47a3-8d49-78697397c304 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:23.911583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:23.912378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:23.912656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:23.912924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:23.913565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:23.915071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:23.927378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:23.928399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 10 13:58:23.929273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:23.936932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 503/999] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:23.937070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de0b9-093b-485a-9150-9423411a3273 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:24.034363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de0b9-093b-485a-9150-9423411a3273 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:24.068746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:24.069410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:24.069690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:24.069962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:24.070697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] RBAC: Adding query filter params (name=compute) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:24.071586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:24.073484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:24.082385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 497/1000] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:24.088354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:24.190982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a7188-f9d9-4c44-9736-7ae108a051c2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:24.228709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:24.229383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:24.229662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:24.229999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:24.230745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:24.231093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:24.232463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffa6f-fa93-4029-bdf5-943a8984fab2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:24.243825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 504/1001] 10.222.0.58 () {56 vars in 1112 bytes} [Tue May 10 13:58:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:24.244070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:24.341403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eeab8-52b6-4bc1-824e-c1b460360cce None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:24.374700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:24.375357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:24.375635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:24.375908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:24.376643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:24.377996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f89ef-c48f-4b9b-ac70-9c2e9b907bee None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:24.471780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 498/1002] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:24.759205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095d4dd-8b01-453e-9c7c-9cd7d185edca None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:24.759511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095d4dd-8b01-453e-9c7c-9cd7d185edca None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:24.759682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095d4dd-8b01-453e-9c7c-9cd7d185edca None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:24.761073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 505/1003] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:24.766924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:24.767210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:24.767485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:24.784822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:24.785997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:24.866809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 499/1004] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:24 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:24.890811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095d4dd-8b01-453e-9c7c-9cd7d185edca None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:25.042706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095d4dd-8b01-453e-9c7c-9cd7d185edca None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:25.086974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:25.087776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.088144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.088466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.089125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:25.090528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:25.102328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:25.103436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:25.104304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:25.108848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 506/1005] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 10 13:58:24 2022] GET /identity/v3/services/image => generated 86 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:25.109872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:25.231567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4b80e-df7e-48b7-ba27-6f070f6c9b03 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:25.267717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:25.268386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.268665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.268938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.269702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] RBAC: Adding query filter params (name=image) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:25.270006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:25.271449 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d2690-e75c-48fa-9068-4494fee0d57e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:25.281294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 500/1006] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:25.282667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:25.378025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da76542-a1aa-4126-99a8-28defe1dedd0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:25.410821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:25.411487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.411765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.412035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.412732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] RBAC: Adding query filter params (type=image) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:25.413033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:25.414425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea343-9362-4dfa-8947-76ac3a89fe90 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:25.418985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 507/1007] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:25.678331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a59b-e2e2-4a9b-ba8a-475f680e22f9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.678605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a59b-e2e2-4a9b-ba8a-475f680e22f9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.678886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a59b-e2e2-4a9b-ba8a-475f680e22f9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.680335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 501/1008] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:25.685757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.686046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.686361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.705306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:25.707047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:25.778906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 508/1009] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:25 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:25.801383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a59b-e2e2-4a9b-ba8a-475f680e22f9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:25.902764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a59b-e2e2-4a9b-ba8a-475f680e22f9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:25.936856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:25.938005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:25.938513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:25.939012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:25.939914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:25.941502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:25.954129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:25.955257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 10 13:58:25.956084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:25.956840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 502/1010] 10.222.0.58 () {56 vars in 1126 bytes} [Tue May 10 13:58:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:25.962928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:26.058366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efacfd-d6c3-4d1f-9126-48aaa86ea95a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:26.091016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:26.091772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:26.091963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:26.092237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:26.092941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:26.093251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:26.094654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:26.106256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 509/1011] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:26.106417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:26.209234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1bfe9-129f-4f29-a60f-cd04c34d094d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:26.250337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:26.251009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:26.251290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:26.251561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:26.252261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:26.252566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:26.253927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e3d8f-809d-4a76-bb88-e6e695a68aab None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:26.259748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 503/1012] 10.222.0.58 () {56 vars in 1122 bytes} [Tue May 10 13:58:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:26.266089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:26.387530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf624f-e76b-44f3-8949-56ba69a75ed2 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:26.424841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:26.425507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:26.425791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:26.426065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:26.426853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:26.428220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56699f-318e-44a5-80e8-990aeb333b90 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:26.495949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 510/1013] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:26.912360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb78f-aceb-4536-85d6-7761f38b5df5 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:26.912745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb78f-aceb-4536-85d6-7761f38b5df5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:26.912924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb78f-aceb-4536-85d6-7761f38b5df5 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:26.914351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 504/1014] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:26.920522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527e06-055d-4990-9205-899e023baf58 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:26.920814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527e06-055d-4990-9205-899e023baf58 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:26.921089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527e06-055d-4990-9205-899e023baf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:26.938842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c8527e06-055d-4990-9205-899e023baf58 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:26.940022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c8527e06-055d-4990-9205-899e023baf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:27.018456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 511/1015] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:26 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:27.041902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bb78f-aceb-4536-85d6-7761f38b5df5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.147499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bb78f-aceb-4536-85d6-7761f38b5df5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:27.192062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:27.192786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.193064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.193336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.204819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:27.213215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:27.214658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:27.215995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:27.218669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 505/1016] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:27.220381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8527e06-055d-4990-9205-899e023baf58 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.329254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8527e06-055d-4990-9205-899e023baf58 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:27.365074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:27.365751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.366030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.366329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.367083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:27.367384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:27.369693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:27.378476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 512/1017] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:27.384046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.487518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a64c08-0ae1-431d-83fb-6d323751e138 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:27.525444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:27.526288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.526574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.526851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.537900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:27.539416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-4e2e537e-94f2-4d4d-80f5-878357f4cf8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:27.561165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:27.567673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:27.569015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:27.569015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:27.569015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:27.569015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:27.569015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 506/1018] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:27.569306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.672534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025e16b-611d-444e-b8b0-a76b7217d278 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:27.706455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae679-c106-43d3-bcce-b3567555c4c6 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.706953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae679-c106-43d3-bcce-b3567555c4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.707426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae679-c106-43d3-bcce-b3567555c4c6 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.708711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 507/1019] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:27.710913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:27.712179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.712536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.712816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.713576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:27.713883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:27.714740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.715026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.715293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.715448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:27.728568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 513/1020] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:27.734088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.739147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:27.740274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:27.820532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 508/1021] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:27.843858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.856693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830ffb-1af7-43b1-9c05-20ac7de7b203 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:27.891792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:27.892595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:27.892882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:27.893151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:27.900890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:27.902558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:27.918165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:27.919223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:27.920290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:27.923053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 514/1022] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:27.926855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:27.957569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f7cb8-c45f-4368-a0d3-79a62c9c3dce None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.004588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.005330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.005675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.005958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.006730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.009117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.046684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999dbd-21e2-42c0-8aea-0b020ed4f787 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.090867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.091549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.091834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.092110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.093026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:28.093327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.094803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.117027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 515/1023] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:58:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:28.123882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:28.126825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 509/1024] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:27 2022] POST /identity/v3/services => generated 242 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:58:28.238482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bc11-de0a-4c0b-b122-72e67bccf123 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.283792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.284372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.284655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.284927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.360642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.362105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.463024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 516/1025] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:58:28.469657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:28.581272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194cbd-e2d5-4129-bd2a-b655a6d3e54e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.615883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.616667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.616945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.617215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.628670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.630166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:28.647390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:28.648349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:28.649360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:28.649360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:28.649360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:28.649360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:28.654231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 510/1026] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:28.654914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:28.761561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591250-31d3-4fe9-bf46-566763a25a69 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.798864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.799528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.799803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.800080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.800810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:28.801116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.802519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de4a70-d595-4a9f-bbc2-64e00ba93090 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.819008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 517/1027] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:28.821294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:28.854120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebcaa2-cead-4598-9a8b-2af6022fc6c4 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.854376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebcaa2-cead-4598-9a8b-2af6022fc6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.854636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebcaa2-cead-4598-9a8b-2af6022fc6c4 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.856008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 518/1028] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:28.860954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.861242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.861534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.878387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:28.879554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:28.922742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c71821-8695-47ee-961c-896d5f9fcb65 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:28.948087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 519/1029] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:28.957657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:28.958432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:28.958718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:28.958993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:28.966991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:28.968447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:28.975859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:28.984114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:28.985063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:28.985973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:28.990070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 511/1030] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:28.991329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:29.077348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6eac9e-22f2-43e1-b1dc-96cbb72bc3ca None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:29.091977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88b649-9fd5-43ce-bdcb-234238d2864a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:29.111798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:29.112516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:29.112799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:29.113072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:29.114296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:29.115745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:29.117948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 520/1031] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:29.123927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:29.128674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:29.129513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:29.130044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:29.130497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:29.131444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:29.131902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:29.134174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:29.145740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 512/1032] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:29.151825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:29.220910 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42d682-9569-4276-b8bf-7f7deab7113f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:29.250274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa723a9-ebf8-4f6d-a9c3-fa63c6fca2de None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:29.255337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:29.256048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:29.256328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:29.256596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:29.257259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:29.259999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42f51-47a3-432d-809b-6811d47c2079 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:29.287549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:29.288205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:29.296825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:29.296825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:29.296825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] RBAC: Adding query filter params (role.id=4cdecf0e40a34a37af9266d132ea8fd7, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:29.296825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:29.296825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf639-f1d8-4643-ad9e-cf22c2692941 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:29.297638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 513/1033] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:29 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=4cdecf0e40a34a37af9266d132ea8fd7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:29.386241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 521/1034] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:29 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:30.498598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9104a4-57e2-43c5-ad1e-fc3a51abc803 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:30.499191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9104a4-57e2-43c5-ad1e-fc3a51abc803 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:30.499191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9104a4-57e2-43c5-ad1e-fc3a51abc803 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:30.500709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 514/1035] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:30.506043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:30.506538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:30.506733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:30.524984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:30.526175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:30.595840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 522/1036] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:30.618682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9104a4-57e2-43c5-ad1e-fc3a51abc803 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:30.726450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9104a4-57e2-43c5-ad1e-fc3a51abc803 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:30.763443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:30.764148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:30.764435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:30.764710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] PATH_INFO: `/v3/services/image` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:30.765360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:30.766755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:30.778318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:30.779413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:30.780349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:30.782536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 515/1037] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 10 13:58:30 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:30.785464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:30.881856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf1c8f-cdd3-4234-ac3b-b3c5b40abc77 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:30.916361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:30.917050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:30.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:30.917607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:30.918357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] RBAC: Adding query filter params (name=image) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:30.918668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:30.920074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:30.930458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 523/1038] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:30.931641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.049768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a4785-3fe5-4f73-84b6-34c34ea63414 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:31.087984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:31.088652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.088929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.089238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.089921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] RBAC: Adding query filter params (type=image) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:31.090256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:31.091631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:31.102557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 516/1039] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:31.103752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.202423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c18f5-5c5b-4e6a-9503-129526c07a69 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:31.234840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:31.235505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.235780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.236062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.236831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] RBAC: Adding query filter params (interface=public, service_id=67588db1305d499ca0049d8ab5d5b5c8, region_id=RegionOne) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:31.237137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:31.238521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2a4e5-5c37-4457-9b41-4c4afe6674d6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:31.250316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 524/1040] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 13:58:31 2022] GET /identity/v3/endpoints?service_id=67588db1305d499ca0049d8ab5d5b5c8&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:31.250510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.345373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc280-33fe-4c82-aa65-c234b3158a58 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:31.378166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:31.378858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.379137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.379408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.380079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:31.381440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab87ac-9310-4647-91bf-98297de0f935 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:31.394291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 517/1041] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 13:58:31 2022] GET /identity/v3/services => generated 1957 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 13:58:31.611505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d31d-fec7-46be-94df-99d738fd5321 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.612012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d31d-fec7-46be-94df-99d738fd5321 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.612012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d31d-fec7-46be-94df-99d738fd5321 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.613531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 525/1042] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:31.619900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3cf2f-edab-4d0b-a48a-0b6cab3152db None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.620387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3cf2f-edab-4d0b-a48a-0b6cab3152db None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.620489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3cf2f-edab-4d0b-a48a-0b6cab3152db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.638121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-78b3cf2f-edab-4d0b-a48a-0b6cab3152db None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:31.639337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-78b3cf2f-edab-4d0b-a48a-0b6cab3152db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:31.714156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 518/1043] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:31.723829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d7fe-218b-4eff-8a10-3ac5653c8ff1 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.724122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d7fe-218b-4eff-8a10-3ac5653c8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.724403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d7fe-218b-4eff-8a10-3ac5653c8ff1 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.725930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 526/1044] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:31.731748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.732044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.732321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.747848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8d7fe-218b-4eff-8a10-3ac5653c8ff1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.755267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:31.757053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:31.840127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 519/1045] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:31.859528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8d7fe-218b-4eff-8a10-3ac5653c8ff1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:31.865097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.895449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:31.896289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:31.896579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:31.896852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:31.908195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:31.917332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:31.918627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:31.919997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 527/1046] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:31.924614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:31.970601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de65314-27d9-46cf-aa21-b83f78c0dd4d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.007960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.008866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.009226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.009510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] PATH_INFO: `/v3/domains/default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.010922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.012400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.014319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 520/1047] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:32.019679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.033884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42590a9c-5b9f-4eb2-a4d0-657c4bafd8ae None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.073298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.074484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.074987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.075491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.076740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:32.077056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.078516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.089335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 528/1048] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:32.094952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.135236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c85aa-1b02-4223-a339-b56e420fdead None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.170563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.171170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.171457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.171730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.172442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.173877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.199243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b4b9-63b9-47c4-982f-a5a01e1b64b6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.232568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.233348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.233665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.233941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.245179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.246690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:32.264089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:32.265278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:32.271055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:32.271055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:32.271055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:32.271055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:32.271055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 529/1049] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:32.271441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 521/1050] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:58:32 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:58:32.271654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.385397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb5c4c-aab5-4a84-ad3d-e8b742fb53f3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.444530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.445573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.446032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.446506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.447491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:32.447980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.449544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.462758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 530/1051] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:32.468323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.575492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20400b4-1df2-4a47-9be8-7762a40798af None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.618729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.619541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.619828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.620105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.628500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.629943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:32.645323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:32.646632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:32.648119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 522/1052] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:58:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:32.654380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.760611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7c7af-960b-48ad-aa4d-670ef0ed7b0c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.796935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.797606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.797883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.798150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.799025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:32.799334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.800762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.823898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 531/1053] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:58:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:32.825394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:32.927451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0042-0ed1-4b8b-b7cd-26abedd593cb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:32.961860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:32.962935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:32.963371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:32.963825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:32.974797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:32.977042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:32.994877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:32.995949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:32.996848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:32.996848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:32.996848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:32.996848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:32.996848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 523/1054] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:33.003382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:33.120172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6da243-df41-443b-a6a1-b4cf7c841b48 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:33.155681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:33.156633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.157078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.157703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.158650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:33.159121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:33.160783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:33.177646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 532/1055] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:33.179021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:33.308016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f6dc4-bd16-4b3c-b552-b21af6eb6590 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:33.342448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:33.343430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.343806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.344174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.351172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:33.355653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:33.372345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:33.373774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:33.374965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:33.376536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 524/1056] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:33.382099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:33.485018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea9316-962b-48bf-a809-e7286edbd1e9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:33.526378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:33.527054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.527333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.527601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.528391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:33.528692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:33.530124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b10c-7b98-4de3-a5c7-51c26b9d5528 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:33.543803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 533/1057] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:33.549721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:33.651194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d323-1044-432d-8b5d-dc02227fefe1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:33.687833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:33.688476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.688761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.689039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/4cdecf0e40a34a37af9266d132ea8fd7` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.689381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413e30-05d9-478a-afc5-9f0abcfbddbf None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.689814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413e30-05d9-478a-afc5-9f0abcfbddbf None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.689979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413e30-05d9-478a-afc5-9f0abcfbddbf None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.691528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=ffdd6ee2e1e64d2fb0d6d05687df82c1, role_id=4cdecf0e40a34a37af9266d132ea8fd7)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:33.691655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 534/1058] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 13:58:33.692787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:33.697526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.697794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.698141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.719312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:33.720543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:33.755902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 525/1059] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:33 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/ffdd6ee2e1e64d2fb0d6d05687df82c1/roles/4cdecf0e40a34a37af9266d132ea8fd7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:58:33.803815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 535/1060] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:33.831026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:33.938626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c6f6c-a8a9-45fb-8c8e-bc96fe1bc67e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:33.976233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:33.977125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:33.977451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:33.977728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:33.978509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:33.979926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:33.991519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:33.992896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:33.993995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 526/1061] 10.222.0.58 () {56 vars in 1105 bytes} [Tue May 10 13:58:33 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:33.999815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:34.115383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846d404-fc53-46ac-86f0-f06bb7d6dea3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:34.149721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:34.150427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.150708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.150987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.151702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] RBAC: Adding query filter params (name=image) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:34.152152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:34.154491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:34.159805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 536/1062] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:34.166620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:34.264780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60976-0b86-4674-8abe-e4f87f117da1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:34.297921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:34.298615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.298892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.299164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.299874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:34.300173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:34.301519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0cee-8fe1-4755-9c8c-b4eedfd3c2d2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:34.306909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 527/1063] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:34.313112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:34.416691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbfde-25ba-436d-8395-a5e14667990c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:34.458429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:34.459460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.459893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.460366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.461333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:34.462967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48954b6-8cbc-48c4-b45a-e578cc6d8c6b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:34.540139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 537/1064] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 13:58:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:34.627256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b1c-7124-425e-abe8-2064a2d918b3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.627557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b1c-7124-425e-abe8-2064a2d918b3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.627826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b1c-7124-425e-abe8-2064a2d918b3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.629446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 528/1065] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:34.635814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.636107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.636391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.654968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:34.656198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:34.739679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 538/1066] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:34.774511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f887b1c-7124-425e-abe8-2064a2d918b3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:34.883148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f887b1c-7124-425e-abe8-2064a2d918b3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:34.919843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:34.920645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:34.920932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:34.921209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:34.932186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:34.940158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:34.941314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:34.942215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 529/1067] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:34.948474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:35.053613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183c5c6-bd64-4cab-ad9c-7b1f15f1dcfd None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:35.090009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:35.090955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:35.091238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:35.091515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:35.092278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:35.092590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:35.093982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:35.109953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 539/1068] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:35.111574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:35.235706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91820cc-a7f5-4d24-8be5-07d3a7f93f36 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:35.270118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:35.271255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:35.271717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:35.272152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] PATH_INFO: `/v3/users/b4fba56211df48bb8b980f25e5e83131` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:35.286135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] RBAC: Authorizing `identity:get_user(user_id=b4fba56211df48bb8b980f25e5e83131)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:35.287849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:35.294765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 530/1069] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:35 2022] GET /identity/v3/users/b4fba56211df48bb8b980f25e5e83131 => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:35.296005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:35.400559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cdfc6-0c65-4b6f-8d76-f3f0a7ed4956 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:35.434830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:35.435742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:35.436180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:35.436610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:35.508716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:35.510547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:35.611556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 540/1070] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:35.612773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:35.713646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a6b6-e82b-4a16-a835-5584effbf15a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:35.751442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:35.752300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:35.752580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:35.752852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:35.754018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:35.755467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e913-e98e-4070-bec7-9730704bfbc9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:35.762500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 531/1071] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:35 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:35.763758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:35.873327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c8d67-ba76-43ff-82cb-599b48a450de None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:35.912605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:35.913277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:35.913569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:35.914438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:35.915442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=259a884263154d75b14d10d80a7b4b62, user.id=b4fba56211df48bb8b980f25e5e83131) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:35.915758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:35.917253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4206d-62da-4e67-9b19-f5a966c707e8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:35.934489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 541/1072] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:35 2022] GET /identity/v3/role_assignments?user.id=b4fba56211df48bb8b980f25e5e83131&scope.project.id=259a884263154d75b14d10d80a7b4b62&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:36.084183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280a8-566f-48a9-9e9e-f497140da621 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.084699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280a8-566f-48a9-9e9e-f497140da621 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.084699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280a8-566f-48a9-9e9e-f497140da621 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.086443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 532/1073] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:36.092680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75982522-4775-467c-841f-6143d22fc895 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.092977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75982522-4775-467c-841f-6143d22fc895 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.093245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75982522-4775-467c-841f-6143d22fc895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.111089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-75982522-4775-467c-841f-6143d22fc895 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:36.112208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-75982522-4775-467c-841f-6143d22fc895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:36.180036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 542/1074] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:36.202611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b280a8-566f-48a9-9e9e-f497140da621 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:36.306078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b280a8-566f-48a9-9e9e-f497140da621 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:36.339908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:36.340647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.340929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.341205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.351999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:36.360249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:36.361189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:36.367223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 533/1075] 10.222.0.58 () {56 vars in 1120 bytes} [Tue May 10 13:58:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:36.369385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75982522-4775-467c-841f-6143d22fc895 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:36.467634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75982522-4775-467c-841f-6143d22fc895 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:36.501707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:36.502422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.502708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.502988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.503732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:36.504041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:36.505410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:36.518234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 543/1076] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:58:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:36.522599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:36.625384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa16691-202b-4d0b-b96d-f656af460144 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:36.659397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:36.660193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.660476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.660750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.671971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:36.673421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:36.691071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:36.692209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:36.693122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:36.693122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:36.693122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:36.693122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:36.693122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 534/1077] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:36.699804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:36.809746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3ee3-b3b2-48ae-aa1f-27f18915c4d4 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:36.842001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:36.842717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.843003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.843281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.844006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:36.844317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:36.845688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4155b47-c6ef-413a-a69d-4e4929ff4723 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:36.862974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 544/1078] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:36.864387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:36.946064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f260f-a77c-4bd4-b6b4-9dd56a998e94 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.946293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f260f-a77c-4bd4-b6b4-9dd56a998e94 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.946465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f260f-a77c-4bd4-b6b4-9dd56a998e94 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.947777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 545/1079] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:36.952813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:36.953134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:36.953366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:36.967503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95bd1c-9116-4ee0-871f-137efc814845 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:36.970538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:36.971718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:37.002593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.003391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.003718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.003999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.011783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.013243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 13:58:37.028264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:37.029254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 10 13:58:37.030181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:37.034157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 535/1080] 10.222.0.58 () {56 vars in 1093 bytes} [Tue May 10 13:58:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:37.034975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.040693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 546/1081] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:37.063466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.149888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c4f32-918e-4a11-ba23-d4bf9aa9edc1 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.163909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a254bd-3037-44d5-a190-0f7c55ac50b0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.183497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.184176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.184460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.184734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.185563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:37.185873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.187313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.206998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.207574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.207855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.208127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.216230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 536/1082] 10.222.0.58 () {56 vars in 1133 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:37.221828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.299764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.302978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.371681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbaa2e-877c-4274-add6-7df2fbea6a13 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.410221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.410972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.411196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.411445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.438396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 547/1083] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:58:37.439396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.485809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.487204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.549481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7e900-4303-4c57-a306-81f58d85f39d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.592848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.593656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.593932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.594230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.606290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.608101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.616951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 537/1084] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:58:37.617538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:37.628407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:37.630881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:37.631957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:37.631957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:37.631957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:37.631957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:37.631957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 548/1085] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:37.636342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.735593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26d991-74fa-4b4e-adaa-2a63f66d5c0c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.743626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6edbd-f314-42e2-8cbc-de33c12c0163 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.770490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.771283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.771569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.771842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.783386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.783875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.784259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.784563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.784843 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.785581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:37.785891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.786728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.787313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.799928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 549/1086] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:58:37.812334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:37.813345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:37.814905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 13:58:37.814905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:58:37.814905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 13:58:37.814905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:37.814905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 538/1087] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:37.819533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:37.941968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621bbc-7db0-4aa2-8124-0c2353c3df25 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:37.978905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:37.979624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:37.979894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:37.980170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:37.980930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:37.981238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:37.982721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474ffd0-b239-4db8-ae82-6366ee954b8c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:37.997481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 550/1088] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 13:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:38.003797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.111814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cda47-fe25-430f-a2d8-90e47352c1ef None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.148557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.149692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.149974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.150653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.156974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.158444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 13:58:38.172573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:38.173777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:58:38.175307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 539/1089] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:58:38.175918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc7aa-e1c2-4b7f-af5b-575c1e95c397 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.176015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc7aa-e1c2-4b7f-af5b-575c1e95c397 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.176214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc7aa-e1c2-4b7f-af5b-575c1e95c397 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.177568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 551/1090] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:38.179425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.182565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.182852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.183136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.199686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:38.200870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:38.267122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 552/1091] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:58:38.276711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8113c6a-1aae-4821-ad9a-4b40a0a5a8cd None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.290912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.309487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.310148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.310461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.310734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.311511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:38.311822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.313220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.325215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 540/1092] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:38.331197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.386591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4514d-dc10-4c83-b430-6429561dfb9d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.419653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.420442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.420724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.420997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.424905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd58fa-dcb4-4b4e-9b35-c369a09ccba6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.431610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:38.439194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:38.440236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:38.441081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:38.445500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 553/1093] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:38.446416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.459039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.459702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.459985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.460276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.461191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] RBAC: Adding query filter params (role.id=4cdecf0e40a34a37af9266d132ea8fd7, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:38.461504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.462998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.468896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 541/1094] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/role_assignments?user.id=ffdd6ee2e1e64d2fb0d6d05687df82c1&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=4cdecf0e40a34a37af9266d132ea8fd7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:38.553416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6bcb4-904c-400e-83fe-4858e11196ff None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.589760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.590466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.590746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.591017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.591775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:38.592079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.593466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.603756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 554/1095] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:58:38.609763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.714811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853b0aa-0964-4a8b-904d-cb282eaf6267 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.750024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.750927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.751209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.751483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] PATH_INFO: `/v3/users/b4fba56211df48bb8b980f25e5e83131` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.752667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] RBAC: Authorizing `identity:get_user(user_id=b4fba56211df48bb8b980f25e5e83131)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.754251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.756209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 542/1096] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/users/b4fba56211df48bb8b980f25e5e83131 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:38.761796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:38.867215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bc4d-79d6-4124-9361-c4b13a4eabac None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:38.904969 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:38.907287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:38.907287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:38.907287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:38.907737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:38.909169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4d877-dc56-447a-b776-3deed5c7085f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:38.911370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 555/1097] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:38 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:38.918386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:39.027841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d210c-e6e2-4138-b7ee-c1ec13deec8d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:39.062748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:39.063626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:39.063906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:39.064178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62/users/b4fba56211df48bb8b980f25e5e83131/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:39.066136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] RBAC: Authorizing `identity:create_grant(project_id=259a884263154d75b14d10d80a7b4b62, user_id=b4fba56211df48bb8b980f25e5e83131, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:39.067573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f8ef6-4756-4607-812d-9a2273bd2a4b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:39.138570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 543/1098] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:38 2022] PUT /identity/v3/projects/259a884263154d75b14d10d80a7b4b62/users/b4fba56211df48bb8b980f25e5e83131/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:58:41.376378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076427-43e3-41b3-8b1f-6706968560a9 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:41.376967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076427-43e3-41b3-8b1f-6706968560a9 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:41.376967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076427-43e3-41b3-8b1f-6706968560a9 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:41.378323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 556/1099] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:41.384190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:41.384604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:41.384745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:41.402211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:41.403498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:41.469067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 544/1100] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:41.493097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076427-43e3-41b3-8b1f-6706968560a9 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:41.592343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076427-43e3-41b3-8b1f-6706968560a9 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:41.626212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:41.627064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:41.627291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:41.627562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:41.637698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:58:41.645237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:58:41.646493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:58:41.650927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 557/1101] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:58:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 13:58:41.655196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:41.755422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121c7f0-7dab-490b-aae7-6a6b00f9d358 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:41.789053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:41.789722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:41.790004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:41.790307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:41.791048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:41.791352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:41.792747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:41.800611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 545/1102] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:58:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:41.807739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:41.909745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d922-b24f-4b6f-aa2d-c61c3e60102e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:41.943004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:41.943894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:41.944174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:41.944543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] PATH_INFO: `/v3/users/b4fba56211df48bb8b980f25e5e83131` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:41.945741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] RBAC: Authorizing `identity:get_user(user_id=b4fba56211df48bb8b980f25e5e83131)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:41.947203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:41.948846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 558/1103] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:41 2022] GET /identity/v3/users/b4fba56211df48bb8b980f25e5e83131 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:41.956285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:42.054037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a846a0-4e79-4e55-8c09-b9a41c9db77e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:42.087537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:42.088184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:42.088466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:42.088741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:42.154248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:42.155692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:42.251549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 546/1104] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 10 13:58:42.257495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:42.356447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15cfe-11ec-4dbc-8efd-eeface889a8c None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:42.387843 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:42.388698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:42.388979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:42.389370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:42.390573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:42.392002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666b46-9962-4303-ae80-e629949c38ea None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:42.393707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 559/1105] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:42 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:42.400678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:42.495489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f177639-052f-4872-a5ac-d5782d229341 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:42.527998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:42.528573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:42.528851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:42.529120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:42.529930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=259a884263154d75b14d10d80a7b4b62, user.id=b4fba56211df48bb8b980f25e5e83131) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:42.530266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:42.531666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f17fe9-1c7f-47e9-a8de-d816b6ed4ad2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:42.537221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 547/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:42 2022] GET /identity/v3/role_assignments?user.id=b4fba56211df48bb8b980f25e5e83131&scope.project.id=259a884263154d75b14d10d80a7b4b62&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:44.792367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d72dd-8059-49b2-a9b8-a18d3ad6b5f8 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:44.792916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d72dd-8059-49b2-a9b8-a18d3ad6b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:44.792916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d72dd-8059-49b2-a9b8-a18d3ad6b5f8 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:44.794309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 560/1107] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:44.799600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:44.800036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:44.800156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:44.816851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:44.818010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:44.882766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 548/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:44.907080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d72dd-8059-49b2-a9b8-a18d3ad6b5f8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:45.007905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d72dd-8059-49b2-a9b8-a18d3ad6b5f8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:45.039782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:45.040559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:45.040841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:45.041116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:45.042437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:45.043874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7c26a-b77c-464e-be4d-3995b20980ab None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:45.045491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 561/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:45.051247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:45.145800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85f422-fe7e-47a6-a2ee-ca08c82f4366 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:45.179169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:45.179828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:45.180112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:45.180388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:45.181100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:45.182555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb915a4-7e88-42cf-865d-671a10733f95 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:45.297761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 549/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:58:45 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:47.582786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39bd7-e3be-4f2d-a71b-e803a1d0b4c1 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:47.583356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39bd7-e3be-4f2d-a71b-e803a1d0b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:47.583356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39bd7-e3be-4f2d-a71b-e803a1d0b4c1 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:47.585258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 562/1111] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:47.590510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:47.590904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:47.591068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:47.608006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:47.609603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:47.683551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 550/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:47.707859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb39bd7-e3be-4f2d-a71b-e803a1d0b4c1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:47.807058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb39bd7-e3be-4f2d-a71b-e803a1d0b4c1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:47.840939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:47.841712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:47.841991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:47.842295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] PATH_INFO: `/v3/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:47.843754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] RBAC: Authorizing `identity:get_role(role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:47.845119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:47.846512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 563/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:47 2022] GET /identity/v3/roles/4642f59bea334153a5500859e246a0c6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:47.852561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:47.947460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1181-f9a9-4515-8368-a0c28329c6ae None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:47.984832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:47.985700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:47.985983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:47.986294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] PATH_INFO: `/v3/users/3656c69ccf4940728ebd770ea811d509` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:47.999811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] RBAC: Authorizing `identity:get_user(user_id=3656c69ccf4940728ebd770ea811d509)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:48.001257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:48.002982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 551/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:47 2022] GET /identity/v3/users/3656c69ccf4940728ebd770ea811d509 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:48.009014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:48.103323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e61b07-d4e9-4681-854c-b10c1030c6f7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:48.136180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:48.136822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:48.137106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:48.137381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:48.200800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:48.202266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:48.293762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 564/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:48.299822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:48.394329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2728-e770-4fcb-89b2-3566739a2bfd None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:48.425743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:48.426615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:48.426902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:48.427180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:48.428866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:48.430422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0e01d-8d3e-4d66-ac10-c9490ddb1173 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:48.432117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 552/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:48 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:48.438249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:48.533552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91685a-b617-440a-a167-2da8ad2a96da None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:48.565682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:48.566366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:48.566647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:48.566922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:48.567747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=259a884263154d75b14d10d80a7b4b62, user.id=3656c69ccf4940728ebd770ea811d509) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:48.568061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:48.569525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8681f4-9252-49e0-aff9-c462b0e7038d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:48.574891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 565/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:48 2022] GET /identity/v3/role_assignments?user.id=3656c69ccf4940728ebd770ea811d509&scope.project.id=259a884263154d75b14d10d80a7b4b62&role.id=4642f59bea334153a5500859e246a0c6 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:50.777202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010be54-417e-42a2-8c0b-71ef7526e09f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:50.777764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010be54-417e-42a2-8c0b-71ef7526e09f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:50.777764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010be54-417e-42a2-8c0b-71ef7526e09f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:50.779168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 553/1118] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:50.784480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:50.784899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:50.785035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:50.801806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:50.803006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:50.866011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 566/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:50.889456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010be54-417e-42a2-8c0b-71ef7526e09f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:50.985824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010be54-417e-42a2-8c0b-71ef7526e09f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:51.019475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:51.020330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:51.020616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:51.020894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] PATH_INFO: `/v3/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:51.022588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] RBAC: Authorizing `identity:get_role(role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:51.023964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:51.025373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 554/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:50 2022] GET /identity/v3/roles/4642f59bea334153a5500859e246a0c6 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:51.031289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:51.129953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65950a-2985-48f7-8da0-2681330f8fe8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:51.162079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:51.163530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:51.163817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:51.164095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] PATH_INFO: `/v3/users/3656c69ccf4940728ebd770ea811d509` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:51.165284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] RBAC: Authorizing `identity:get_user(user_id=3656c69ccf4940728ebd770ea811d509)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:51.166764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:51.168403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 567/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:51 2022] GET /identity/v3/users/3656c69ccf4940728ebd770ea811d509 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:51.174106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:51.271390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ad727-c943-4864-83d6-5b62f37eda3e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:51.304714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:51.305557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:51.305837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:51.306108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:51.307293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:51.308695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebb00b-2c66-40c7-a66e-bf983a43f1aa None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:51.310384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 555/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:51 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:51.316206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:51.417877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449db6-928f-4e40-bed4-5d2d7627db21 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:51.450787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:51.451657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:51.451934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:51.452202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62/users/3656c69ccf4940728ebd770ea811d509/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:51.454384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] RBAC: Authorizing `identity:create_grant(project_id=259a884263154d75b14d10d80a7b4b62, user_id=3656c69ccf4940728ebd770ea811d509, role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:51.455810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20547ba2-ef3a-474d-b684-561810cb4ef3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:51.526642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 568/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:58:51 2022] PUT /identity/v3/projects/259a884263154d75b14d10d80a7b4b62/users/3656c69ccf4940728ebd770ea811d509/roles/4642f59bea334153a5500859e246a0c6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:58:53.741992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809027fa-a105-4a38-b7e2-d3779b85fe05 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:53.742483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809027fa-a105-4a38-b7e2-d3779b85fe05 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:53.742600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809027fa-a105-4a38-b7e2-d3779b85fe05 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:53.743952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 556/1124] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:53.749696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:53.750102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:53.750270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:53.766430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:53.767595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:53.833745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 569/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:53.858430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809027fa-a105-4a38-b7e2-d3779b85fe05 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:53.957084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809027fa-a105-4a38-b7e2-d3779b85fe05 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:53.989589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:53.990871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:53.992593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:53.992879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] PATH_INFO: `/v3/roles/4642f59bea334153a5500859e246a0c6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:53.994420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] RBAC: Authorizing `identity:get_role(role_id=4642f59bea334153a5500859e246a0c6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:53.995789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:53.997155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 557/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:53 2022] GET /identity/v3/roles/4642f59bea334153a5500859e246a0c6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:54.003272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:54.102829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93c5db-f04a-480d-a35e-67f8f3908ed4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:54.136378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:54.137231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:54.137509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:54.137781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] PATH_INFO: `/v3/users/3656c69ccf4940728ebd770ea811d509` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:54.138971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] RBAC: Authorizing `identity:get_user(user_id=3656c69ccf4940728ebd770ea811d509)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:54.140403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:54.142008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 570/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:58:54 2022] GET /identity/v3/users/3656c69ccf4940728ebd770ea811d509 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:54.147923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:54.241933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8e5c8-1fcf-446a-8235-4c53da8eacc9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:54.274214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:54.274850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:54.275127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:54.275399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:54.338024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:54.339467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:54.436741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 558/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:58:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:58:54.442680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:54.536387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747443d5-14f6-4d56-a135-55f6ad0f16c6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:54.569475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:54.570334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:54.570618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:54.570891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] PATH_INFO: `/v3/projects/259a884263154d75b14d10d80a7b4b62` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:54.572052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] RBAC: Authorizing `identity:get_project(project_id=259a884263154d75b14d10d80a7b4b62)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:54.573477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bcc6b-e36a-43ed-9630-fef86c5dd09c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:54.575171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 571/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:58:54 2022] GET /identity/v3/projects/259a884263154d75b14d10d80a7b4b62 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:54.581381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:54.677096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f993ed-c199-415f-8411-b8a464d31e81 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:54.711021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:54.711793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:54.712075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:54.712485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:54.713327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] RBAC: Adding query filter params (role.id=4642f59bea334153a5500859e246a0c6, scope.project.id=259a884263154d75b14d10d80a7b4b62, user.id=3656c69ccf4940728ebd770ea811d509) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:58:54.713327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:54.713327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f107b-0e6c-40ad-aa7f-1a2b79df6c4a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:54.720990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 559/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:58:54 2022] GET /identity/v3/role_assignments?user.id=3656c69ccf4940728ebd770ea811d509&scope.project.id=259a884263154d75b14d10d80a7b4b62&role.id=4642f59bea334153a5500859e246a0c6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:56.970145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c112c8-5e2a-4762-a5f8-cad51b1f7473 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:56.970688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c112c8-5e2a-4762-a5f8-cad51b1f7473 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:56.970886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c112c8-5e2a-4762-a5f8-cad51b1f7473 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:56.972143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 572/1131] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:56.978303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:56.978782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:56.978893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:56.995811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:56.996990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:57.060612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 560/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:57.084202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c112c8-5e2a-4762-a5f8-cad51b1f7473 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:57.184222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c112c8-5e2a-4762-a5f8-cad51b1f7473 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:57.216711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:57.217495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:57.217778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:57.218053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:57.219249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:57.220685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e7f90-f7c5-4c10-af38-7b2daed922a2 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:57.222385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 573/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:58:57.228934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:57.333452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63016f3-cf10-41fd-8240-8ed3511f9ccd None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:57.371595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:57.372539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:57.372819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:57.373095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:57.373801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:57.375276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160b209-35c0-4bcd-9721-c25a3fb8183e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:58:57.464747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 561/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:58:57 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:58:59.750364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8187f-494e-475c-80df-1d73061bf47b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:59.750871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8187f-494e-475c-80df-1d73061bf47b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:59.750982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8187f-494e-475c-80df-1d73061bf47b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:59.752281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 574/1135] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:58:59.758058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:59.758507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:59.758643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:59.775790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:58:59.776972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:58:59.840952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 562/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:58:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:58:59.865655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8187f-494e-475c-80df-1d73061bf47b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:58:59.962919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8187f-494e-475c-80df-1d73061bf47b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:58:59.994536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:58:59.995255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:58:59.995537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:58:59.995810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:58:59.996978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:58:59.998717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe081e-4b50-4200-b58f-74033b8d508a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:00.000318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 575/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 13:58:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:00.007065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:00.101364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcf2dd-6c2c-48db-b099-53bd6a6a886a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:00.141323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:00.141994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:00.142302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:00.142577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:00.143271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:00.145047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a44c53-47bb-44f1-8dbb-7bcaa73739c8 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:00.245154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 563/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:59:00 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:59:02.513048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437738e-b424-4499-886f-905df9b4301d None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:02.513619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437738e-b424-4499-886f-905df9b4301d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:02.513619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437738e-b424-4499-886f-905df9b4301d None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:02.515005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 576/1139] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:02.520173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:02.520559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:02.520728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:02.537165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:02.538377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:02.602390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 564/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:02.626157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437738e-b424-4499-886f-905df9b4301d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:02.726674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437738e-b424-4499-886f-905df9b4301d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:02.762552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:02.763395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:02.763621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:02.763894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:02.773726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:02.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:02.784021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:02.785023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 577/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:02.790890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:02.893397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01342ea1-d997-4a1d-8c2b-0e7f8e087898 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:02.927310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:02.927939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:02.928221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:02.928500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:02.929242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:02.929552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:02.930946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:02.938905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 565/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:02.945301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:03.058030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf287-f266-4ac7-a1a1-acd48d7971d1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:03.092474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:03.093349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:03.093632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:03.093908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] PATH_INFO: `/v3/users/4c059967b2b44901922a78552edfa4c2` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:03.111037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] RBAC: Authorizing `identity:get_user(user_id=4c059967b2b44901922a78552edfa4c2)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:03.112497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:03.114465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 578/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:02 2022] GET /identity/v3/users/4c059967b2b44901922a78552edfa4c2 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:03.121498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:03.228973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4be2-969c-4855-9f64-08bec021ccc2 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:03.262719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:03.263361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:03.263640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:03.263910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:03.329003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:03.330390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:03.437931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 566/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:59:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:59:03.443867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:03.542543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297330d-a411-46b7-b794-5294687ee503 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:03.577209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:03.578086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:03.578400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:03.578668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] PATH_INFO: `/v3/projects/2d1dd21ff34d48b1aee892001aca78ba` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:03.579879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] RBAC: Authorizing `identity:get_project(project_id=2d1dd21ff34d48b1aee892001aca78ba)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:03.581250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a19fca-c2d6-40aa-9064-1370a5022038 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:03.582957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 579/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:03 2022] GET /identity/v3/projects/2d1dd21ff34d48b1aee892001aca78ba => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:03.589162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:03.684939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad53207-39a6-47b6-b369-4d3950b5ce02 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:03.717309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:03.718004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:03.718327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:03.718607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:03.719434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=2d1dd21ff34d48b1aee892001aca78ba, user.id=4c059967b2b44901922a78552edfa4c2) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:03.719747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:03.721177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bdfd8-0940-4325-ae47-d3a4fbfcce4d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:03.727450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 567/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:59:03 2022] GET /identity/v3/role_assignments?user.id=4c059967b2b44901922a78552edfa4c2&scope.project.id=2d1dd21ff34d48b1aee892001aca78ba&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:05.945156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fd880-b81f-47cf-be45-86ce482552ca None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:05.945708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fd880-b81f-47cf-be45-86ce482552ca None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:05.945822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fd880-b81f-47cf-be45-86ce482552ca None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:05.947242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 580/1147] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:05.952410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:05.952852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:05.952954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:05.969831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:05.971071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:06.033944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 568/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:06.056183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fd880-b81f-47cf-be45-86ce482552ca None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:06.157240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fd880-b81f-47cf-be45-86ce482552ca None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:06.194554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:06.195348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:06.195629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:06.195901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:06.205681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:06.213939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:06.215629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:06.216756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 581/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:06.221307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:06.315588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231971d-12aa-4cba-998c-4ff30d5c014c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:06.347820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:06.348481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:06.348812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:06.349095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:06.349843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:06.350144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:06.351529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:06.359311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 569/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:06.365816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:06.461879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4443a-499d-4f1a-be29-eab9885ebb54 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:06.494780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:06.495579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:06.495865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:06.496143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] PATH_INFO: `/v3/users/4c059967b2b44901922a78552edfa4c2` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:06.497606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] RBAC: Authorizing `identity:get_user(user_id=4c059967b2b44901922a78552edfa4c2)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:06.499068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:06.500702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 582/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:06 2022] GET /identity/v3/users/4c059967b2b44901922a78552edfa4c2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:06.507376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:06.603324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82edf1f-f088-4cc1-a23e-2cd2eaf77511 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:06.635581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:06.636430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:06.636711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:06.636985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] PATH_INFO: `/v3/projects/2d1dd21ff34d48b1aee892001aca78ba` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:06.638139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] RBAC: Authorizing `identity:get_project(project_id=2d1dd21ff34d48b1aee892001aca78ba)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:06.639587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42812-ba76-4ec1-81d4-7becbf70f53a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:06.641275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 570/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:06 2022] GET /identity/v3/projects/2d1dd21ff34d48b1aee892001aca78ba => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 13:59:06.647561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:06.746171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ffd-d446-4d09-8721-89c12ab46e9d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:06.778528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:06.779416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:06.779706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:06.779981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] PATH_INFO: `/v3/projects/2d1dd21ff34d48b1aee892001aca78ba/users/4c059967b2b44901922a78552edfa4c2/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:06.781946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] RBAC: Authorizing `identity:create_grant(project_id=2d1dd21ff34d48b1aee892001aca78ba, user_id=4c059967b2b44901922a78552edfa4c2, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:06.783608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3db87-044f-45a5-bb15-93104cde2e2b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:06.851394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 583/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:59:06 2022] PUT /identity/v3/projects/2d1dd21ff34d48b1aee892001aca78ba/users/4c059967b2b44901922a78552edfa4c2/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:59:09.122923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d039e90-ac21-44f1-abea-f4a05d32aadc None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.123446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d039e90-ac21-44f1-abea-f4a05d32aadc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.123446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d039e90-ac21-44f1-abea-f4a05d32aadc None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.124769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 571/1154] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:09.130924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.131396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.131563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.148619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:09.149849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:09.216539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 584/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:09.240533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d039e90-ac21-44f1-abea-f4a05d32aadc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:09.341649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d039e90-ac21-44f1-abea-f4a05d32aadc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:09.377548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:09.378368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.378657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.378934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.389039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:09.397195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:09.398173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:59:09.399062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 572/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:09.404473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:09.520174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91893629-0ae4-4b81-9755-e60f58255ae1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:09.554498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:09.555156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.555436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.555708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.556451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:09.556753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:09.558308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:09.566444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 585/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:09.572717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:09.678489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160d68c-0702-40bd-b08b-c573e7863468 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:09.711463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:09.712415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.712694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.712972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] PATH_INFO: `/v3/users/4c059967b2b44901922a78552edfa4c2` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.714143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] RBAC: Authorizing `identity:get_user(user_id=4c059967b2b44901922a78552edfa4c2)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:09.715694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:09.717329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 573/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:09 2022] GET /identity/v3/users/4c059967b2b44901922a78552edfa4c2 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:09.724890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:09.845717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05358c-7cd3-436c-b1e9-109b2508f38b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:09.882100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:09.882786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:09.883072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:09.883349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:09.949358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:09.950572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:10.053067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 586/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:59:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:59:10.059079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:10.164803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259818-4b39-4277-80df-8b4c62290c7c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:10.210980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:10.211754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:10.212032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:10.212306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] PATH_INFO: `/v3/projects/2d1dd21ff34d48b1aee892001aca78ba` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:10.213498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=2d1dd21ff34d48b1aee892001aca78ba)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:10.214951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351241e0-8cc7-4542-aaef-682dd27c7ff5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:10.216662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 574/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:10 2022] GET /identity/v3/projects/2d1dd21ff34d48b1aee892001aca78ba => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:10.224970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:10.320821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ca508-1f72-4ef1-895b-8d53a9ba94b6 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:10.352784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:10.353434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:10.353719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:10.353996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:10.354856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=2d1dd21ff34d48b1aee892001aca78ba, user.id=4c059967b2b44901922a78552edfa4c2) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:10.355171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:10.356608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57acb62-d8e1-4e26-b15a-5be99b741dce None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:10.362361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 587/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:59:10 2022] GET /identity/v3/role_assignments?user.id=4c059967b2b44901922a78552edfa4c2&scope.project.id=2d1dd21ff34d48b1aee892001aca78ba&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:12.634962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942170b8-fcdf-401b-b91a-30aef6dbaf6e None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:12.635516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942170b8-fcdf-401b-b91a-30aef6dbaf6e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:12.635516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942170b8-fcdf-401b-b91a-30aef6dbaf6e None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:12.636869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 575/1162] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:12.642303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82719af-bbef-407e-896f-faf3a502178a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:12.642734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82719af-bbef-407e-896f-faf3a502178a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:12.642870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82719af-bbef-407e-896f-faf3a502178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:12.660039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d82719af-bbef-407e-896f-faf3a502178a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:12.661771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d82719af-bbef-407e-896f-faf3a502178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:12.725916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 588/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 13:59:12.748307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942170b8-fcdf-401b-b91a-30aef6dbaf6e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:12.846323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942170b8-fcdf-401b-b91a-30aef6dbaf6e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:12.879701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:12.880519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:12.880797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:12.881067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:12.948181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:12.949524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:13.049813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 576/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:59:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:59:13.061620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82719af-bbef-407e-896f-faf3a502178a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:13.188570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82719af-bbef-407e-896f-faf3a502178a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:13.230133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:13.230954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:13.231237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:13.231514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:13.238280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:13.240354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-ba0f8fe2-a93e-4596-affa-22cf93c8a541 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:13.255450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:13.256755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:13.257731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 589/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 13:59:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:13.263690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:13.358987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c24d-1dff-4778-9305-be6684f49ddb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:13.394091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:13.394785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:13.395068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:13.395875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:13.396594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:13.396900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:13.398287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8a4cc-c4fc-48bf-880d-6d5e248822c1 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:13.405705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 577/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Tue May 10 13:59:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:15.657537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac555de-d37f-48e6-91dc-9f486092a35f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:15.658129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac555de-d37f-48e6-91dc-9f486092a35f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:15.658129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac555de-d37f-48e6-91dc-9f486092a35f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:15.659493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 590/1167] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:15.665704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78531-3064-42c3-bb64-81fd0ee20bb4 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:15.666166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78531-3064-42c3-bb64-81fd0ee20bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:15.666326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c78531-3064-42c3-bb64-81fd0ee20bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:15.684395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e7c78531-3064-42c3-bb64-81fd0ee20bb4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:15.688343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e7c78531-3064-42c3-bb64-81fd0ee20bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:15.752491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 578/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:15.775670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac555de-d37f-48e6-91dc-9f486092a35f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:15.883654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac555de-d37f-48e6-91dc-9f486092a35f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:15.924823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:15.925812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:15.926251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:15.926728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:15.927863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:15.929998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35019875-d926-4cb1-b368-4a5117358426 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:16.034890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 591/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Tue May 10 13:59:15 2022] POST /identity/v3/domains => generated 267 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:59:18.283995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60992-7bb8-47e7-bbb1-91aee53fd5e0 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:18.284565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60992-7bb8-47e7-bbb1-91aee53fd5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:18.284565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60992-7bb8-47e7-bbb1-91aee53fd5e0 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:18.286025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 579/1170] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:18.291286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:18.291699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:18.291844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:18.308998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:18.310186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:18.386478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 592/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:18.409846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b60992-7bb8-47e7-bbb1-91aee53fd5e0 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:18.512875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b60992-7bb8-47e7-bbb1-91aee53fd5e0 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:18.557936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:18.558740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:18.559015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:18.559287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] PATH_INFO: `/v3/domains/45172f2ed3854933ab06c8907502f5fe` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:18.579878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=45172f2ed3854933ab06c8907502f5fe)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:18.581341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a5c74-3b43-41bd-a8c6-61aa772e66cf None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:18.583368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 580/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Tue May 10 13:59:18 2022] GET /identity/v3/domains/45172f2ed3854933ab06c8907502f5fe => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:18.589778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:18.695230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533090e-9e18-4647-bf8b-305e1176844d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:18.728859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:18.729521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:18.729799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:18.730070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:18.730821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:18.732248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fd183-590f-4a95-8508-51f66538979c None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:18.852728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 593/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 13:59:18 2022] POST /identity/v3/projects => generated 361 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 13:59:21.140272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966795-0e77-47ee-91f1-7caeeaa05af9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:21.140821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966795-0e77-47ee-91f1-7caeeaa05af9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:21.140821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a966795-0e77-47ee-91f1-7caeeaa05af9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:21.142190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 581/1174] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:21.147810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:21.148273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:21.148382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:21.165284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:21.166510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:21.232918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 594/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:21.258997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966795-0e77-47ee-91f1-7caeeaa05af9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:21.371129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a966795-0e77-47ee-91f1-7caeeaa05af9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:21.403535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:21.404327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:21.404626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:21.404871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] PATH_INFO: `/v3/domains/45172f2ed3854933ab06c8907502f5fe` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:21.406054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=45172f2ed3854933ab06c8907502f5fe)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:21.407502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c458a0f-3997-4152-9191-a1ea226d2665 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:21.409125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 582/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Tue May 10 13:59:21 2022] GET /identity/v3/domains/45172f2ed3854933ab06c8907502f5fe => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:21.415886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:21.513877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26483e-2c7c-4aa4-99e9-8d47e15573d5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:21.548236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:21.548912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:21.549190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:21.549464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:21.550176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:21.552543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6677d20-1f38-44ae-9308-021a911925b4 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:21.648428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 595/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 13:59:21 2022] POST /identity/v3/users => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 13:59:24.207268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9882ea-9e49-46f5-afe6-dc02b5b9aa65 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.207814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9882ea-9e49-46f5-afe6-dc02b5b9aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.207814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9882ea-9e49-46f5-afe6-dc02b5b9aa65 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:24.209169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 583/1178] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:24.214889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.215422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.215422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:24.232431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:24.233605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:24.299895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 596/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:24.324992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9882ea-9e49-46f5-afe6-dc02b5b9aa65 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:24.439416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9882ea-9e49-46f5-afe6-dc02b5b9aa65 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:24.472644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:24.474673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.475140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.475715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:24.490615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:24.503532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:24.504799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:24.506055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:59:24.507077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 584/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:24.512464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:24.609770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716d1b4-1b3f-4171-95eb-704c555e3184 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:24.642287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:24.642955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.643239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.643517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:24.644259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:24.644565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:24.646694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:24.654448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 597/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:24.661210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:24.758147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63963bc-eda6-4540-a69c-d3ff3e9814fe None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:24.791373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:24.792234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.792515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.792787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] PATH_INFO: `/v3/users/c022f8bfc3a54b0da67fb4bc30683e70` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:24.807553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] RBAC: Authorizing `identity:get_user(user_id=c022f8bfc3a54b0da67fb4bc30683e70)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:24.809008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:24.810762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 585/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:24 2022] GET /identity/v3/users/c022f8bfc3a54b0da67fb4bc30683e70 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:24.816647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:24.912593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb6396-4b73-43de-808f-d8cb8310751d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:24.945180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:24.945828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:24.946104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:24.946527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:25.014752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:25.016120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:25.114448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 598/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:59:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:59:25.120491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:25.217234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8e4f-148c-42d4-8e06-568ff81efce7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:25.251078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:25.251925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:25.252201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:25.252467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] PATH_INFO: `/v3/projects/986697f56c4045a3998a7b4fafc99f1e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:25.253643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] RBAC: Authorizing `identity:get_project(project_id=986697f56c4045a3998a7b4fafc99f1e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:25.255262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a963b-6b94-499a-8397-163e34b97681 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:25.256952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 586/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:25 2022] GET /identity/v3/projects/986697f56c4045a3998a7b4fafc99f1e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:25.263090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:25.370797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701ee-36a8-4d07-aeda-70ddeb7c3ed3 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:25.406499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:25.407158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:25.407440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:25.407713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:25.408538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=986697f56c4045a3998a7b4fafc99f1e, user.id=c022f8bfc3a54b0da67fb4bc30683e70) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:25.408850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:25.410927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf88c55-9117-484b-9824-4a478609ab4a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:25.416478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 599/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:59:25 2022] GET /identity/v3/role_assignments?user.id=c022f8bfc3a54b0da67fb4bc30683e70&scope.project.id=986697f56c4045a3998a7b4fafc99f1e&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:27.707697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f4ece-c4ed-433f-be1e-abb783008260 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:27.708234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f4ece-c4ed-433f-be1e-abb783008260 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:27.708587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f4ece-c4ed-433f-be1e-abb783008260 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:27.710327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 587/1186] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:27.715695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:27.716149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:27.716245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:27.733536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:27.734758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:27.813571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 600/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:27.835807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f4ece-c4ed-433f-be1e-abb783008260 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:27.932971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f4ece-c4ed-433f-be1e-abb783008260 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:27.965048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:27.965761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:27.966038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:27.966332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:27.976084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:27.983502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:27.984593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 13:59:27.985808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 588/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:27.990743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:28.084912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8c58a-1424-40b3-bd8b-31a688437d86 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:28.117264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:28.117927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:28.118235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:28.118562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:28.119259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:28.119566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:28.120931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:28.128934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 601/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:28.135357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:28.231048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5679926-d940-4327-8681-e3f6fae1ca29 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:28.264788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:28.265655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:28.265937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:28.266239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] PATH_INFO: `/v3/users/c022f8bfc3a54b0da67fb4bc30683e70` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:28.267417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] RBAC: Authorizing `identity:get_user(user_id=c022f8bfc3a54b0da67fb4bc30683e70)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:28.268844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:28.270576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 589/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:28 2022] GET /identity/v3/users/c022f8bfc3a54b0da67fb4bc30683e70 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:28.276243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:28.373006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55277b9-4427-41ec-bd05-754b3a27a428 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:28.405614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:28.406503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:28.406789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:28.407067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] PATH_INFO: `/v3/projects/986697f56c4045a3998a7b4fafc99f1e` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:28.408235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] RBAC: Authorizing `identity:get_project(project_id=986697f56c4045a3998a7b4fafc99f1e)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:28.409680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ddd87-0e37-4e46-8c24-07fa8d7b920d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:28.412109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 602/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:28 2022] GET /identity/v3/projects/986697f56c4045a3998a7b4fafc99f1e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:28.418108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:28.511317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf81f8-ccab-4872-96be-42e72bad9c8c None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:28.543323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:28.544204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:28.544487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:28.544764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] PATH_INFO: `/v3/projects/986697f56c4045a3998a7b4fafc99f1e/users/c022f8bfc3a54b0da67fb4bc30683e70/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:28.546718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] RBAC: Authorizing `identity:create_grant(project_id=986697f56c4045a3998a7b4fafc99f1e, user_id=c022f8bfc3a54b0da67fb4bc30683e70, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:28.548142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51561a8-a99c-41aa-8834-4d2fdf46cd8e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:28.638623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 590/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 13:59:28 2022] PUT /identity/v3/projects/986697f56c4045a3998a7b4fafc99f1e/users/c022f8bfc3a54b0da67fb4bc30683e70/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 13:59:30.855787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71cb1-c0d6-4b8b-ac87-1e09b4b9fff0 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:30.856350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71cb1-c0d6-4b8b-ac87-1e09b4b9fff0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:30.856350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71cb1-c0d6-4b8b-ac87-1e09b4b9fff0 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:30.857691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 603/1193] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 13:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 13:59:30.862915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:30.863463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:30.863574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:30.881014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 13:59:30.882237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 13:59:30.946301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 591/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 13:59:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 13:59:30.969461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e71cb1-c0d6-4b8b-ac87-1e09b4b9fff0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.068834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e71cb1-c0d6-4b8b-ac87-1e09b4b9fff0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.100394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.101313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.101596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:31.101867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:31.111479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 13:59:31.119830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 13:59:31.121000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 13:59:31.122547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 604/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 13:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 13:59:31.127303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.222140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf60e4e-7af4-4535-8ff4-1caba2b85ab8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.256094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.256754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.257035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:31.257312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:31.258047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:31.258382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:31.259753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:31.267550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 592/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 13:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:31.274188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.369113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3e55-c0d7-4f52-98ff-0e94f3d1d738 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.400861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.401709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.401987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:31.402283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] PATH_INFO: `/v3/users/c022f8bfc3a54b0da67fb4bc30683e70` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:31.403447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] RBAC: Authorizing `identity:get_user(user_id=c022f8bfc3a54b0da67fb4bc30683e70)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:31.404863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:31.406548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 605/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Tue May 10 13:59:31 2022] GET /identity/v3/users/c022f8bfc3a54b0da67fb4bc30683e70 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:31.412338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.509138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0990-5f46-42a1-b337-d0071a2ed740 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.540818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.541460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.541741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:31.542014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:31.605439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:31.606907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:31.701946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 593/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 13:59:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 13:59:31.709165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.819379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ebee-6966-46aa-ac3d-1d8b9c25574f None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.851577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.852433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.852715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:31.853032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] PATH_INFO: `/v3/projects/986697f56c4045a3998a7b4fafc99f1e` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:31.854293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] RBAC: Authorizing `identity:get_project(project_id=986697f56c4045a3998a7b4fafc99f1e)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:31.855741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a9a6-0a78-45b1-b8a8-858a23abbe41 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:31.857453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 606/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Tue May 10 13:59:31 2022] GET /identity/v3/projects/986697f56c4045a3998a7b4fafc99f1e => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 13:59:31.864547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 13:59:31.964427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9da2c-41a5-44d2-b369-b673f39b63a0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 13:59:31.998793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 13:59:31.999451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 13:59:31.999738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 13:59:32.000008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 13:59:32.000836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=986697f56c4045a3998a7b4fafc99f1e, user.id=c022f8bfc3a54b0da67fb4bc30683e70) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 13:59:32.001142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 13:59:32.002588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552bb64-bf98-4693-8373-a1d41eae14d0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 13:59:32.009674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 594/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 13:59:31 2022] GET /identity/v3/role_assignments?user.id=c022f8bfc3a54b0da67fb4bc30683e70&scope.project.id=986697f56c4045a3998a7b4fafc99f1e&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:47.984954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31727d4-bac3-4699-b168-2d6cc8c1de49 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:47.985522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31727d4-bac3-4699-b168-2d6cc8c1de49 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:47.985522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31727d4-bac3-4699-b168-2d6cc8c1de49 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:47.987448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 607/1201] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:00:47.992577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:47.993015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:47.993117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:48.010546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:00:48.011711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:00:48.077111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 595/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:00:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:00:48.101449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31727d4-bac3-4699-b168-2d6cc8c1de49 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:48.200805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31727d4-bac3-4699-b168-2d6cc8c1de49 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:48.232686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:48.233469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:48.233749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:48.234023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:48.252492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:48.252492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:48.263441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:48.264545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:48.265736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:48.265736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:48.265736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:48.265736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:00:48.265736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 608/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:48.271161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:48.365501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67adc6d-89bb-423a-b987-5e129282cd6c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:48.398124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:48.398807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:48.399087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:48.399361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:48.400075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:48.400383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:48.401739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53379b-286d-4d13-9e97-71332eab5bdd None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:48.413052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 596/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:48.418936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:48.515329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf6cd-b4d8-4cb7-bcaf-63b793fbe6b8 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:48.548083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:48.548742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:48.549020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:48.550024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:48.551008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:48.552569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cf2ce-68cb-458c-ac89-22b3f59307c7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:48.618562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 609/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Tue May 10 14:00:48 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:00:50.875378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba13f-ce3c-4bab-9c08-8c3858e93b71 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:50.875925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba13f-ce3c-4bab-9c08-8c3858e93b71 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:50.875925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deba13f-ce3c-4bab-9c08-8c3858e93b71 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:50.877322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 597/1206] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:00:50.882520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:50.882936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:50.883107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:50.900196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:00:50.901402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:00:50.966052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 610/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:00:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:00:50.989920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba13f-ce3c-4bab-9c08-8c3858e93b71 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.087520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deba13f-ce3c-4bab-9c08-8c3858e93b71 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.119802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.120587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.120867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.121137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.131523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:00:51.139189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:51.140325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:51.141408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 598/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 14:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:51.147204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.239224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5f55d-f5d8-4dd0-8f5a-1e790561fbbf None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.271682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.272274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.272558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.272825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.273558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:51.273857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:51.275234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:51.282656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 611/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:51.288450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.386063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de82ba-a8c2-4e05-bb95-056fd3c65813 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.418319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.419105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.427276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.427276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.432736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:51.435116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:51.461514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:51.462650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:51.463819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:51.463819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:51.463819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:51.463819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:51.463819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 599/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:51.469135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.563907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c96b-b633-4dd0-9b7b-2d42848729c7 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.596912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.597602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.597885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.598160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.598906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:51.599211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:51.600573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:51.611981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 612/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:51.617586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.716006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631419b-69fb-4437-81c9-aeb9e504bee3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.751293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.752150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.752476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.753292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.761602 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:51.765545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:00:51.778529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:51.783786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:51.785175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 600/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:51.785928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:51.883435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb906e1-01e6-4224-84ed-0b9efad5fa8b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:51.915077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:51.915745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:51.916023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:51.916295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:51.917117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:51.917435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:51.918871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:51.935572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 613/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:51.942155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.039773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb3f1-0823-415a-8e69-76eb717b3046 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:52.073297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:52.073944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:52.074245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:52.074523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:52.141191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:52.142584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:52.237468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 601/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 14:00:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:00:52.243322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.340410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d86a9c-fed2-43a1-bc0a-63b2cb7796db None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:52.373823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:52.375026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:52.375308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:52.375586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:52.385902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:52.387382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:52.405010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:52.406088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:52.411500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:52.411500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:52.411500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:52.411500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:00:52.411500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 614/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:00:52.411864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.507147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57cdc-1de8-4856-b844-16856bfd24fb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:52.539242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:52.539900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:52.540184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:52.540461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:52.541274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:52.541581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:52.542974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:52.554455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 602/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:00:52.560198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.658108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69f267-46c8-4aaf-bb62-d6426b14be26 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:52.692953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:52.693806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:52.694081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:52.694388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:52.700928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:52.702408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:00:52.715998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:52.716974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:00:52.718532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 615/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:52.723284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.821614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68626a3-a77f-4377-9946-48b1dc5715ea None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:52.854164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:52.854852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:52.855206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:52.855485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:52.856264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:52.856576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:52.858038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e18f7-f07d-437c-b3bb-cc468a2e611b None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:52.871317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 603/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:52.877806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:52.974779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b97e48-8e71-4f41-9130-423fafaebed0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:53.007551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:53.008202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:53.008486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:53.008758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:53.009652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=639566a1a9de49ef841486e9e21a2116) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:53.009961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:53.011427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9901-6c7e-46b6-9982-7ee490ddbaed None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:53.016860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 616/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 14:00:52 2022] GET /identity/v3/role_assignments?user.id=639566a1a9de49ef841486e9e21a2116&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:55.215805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aafc0-4cc3-42ba-8c7c-2971941b2edf None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.216389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aafc0-4cc3-42ba-8c7c-2971941b2edf None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.216389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aafc0-4cc3-42ba-8c7c-2971941b2edf None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.217791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 604/1220] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:00:55.224099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.224534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.224685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.241842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:00:55.243057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:00:55.308586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 617/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:00:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 10 14:00:55.331540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aafc0-4cc3-42ba-8c7c-2971941b2edf None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:55.430322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aafc0-4cc3-42ba-8c7c-2971941b2edf None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:55.462279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:55.463060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.463339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.463607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.474181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:00:55.481886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:55.483544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:55.484674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 605/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 14:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:00:55.489409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:55.584177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebd6ce-46b0-43f9-90f2-691a3a695cf8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:55.616980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:55.617636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.617922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.618191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.618948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:55.619248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:55.620641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:55.629610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 618/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 14:00:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:55.635484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:55.733480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22aa6-d16a-476f-9f19-202ff04c21fb None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:55.766522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:55.767311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.767589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.767861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.778066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:55.779533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:55.797608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:55.801515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:55.802810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:55.802810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:55.802810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:55.802810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:55.802810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 606/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:55.805379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:55.904624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d7b55-8ed6-4f81-bbd5-541292ee1d44 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:55.937697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:55.938427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:55.938716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:55.938993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:55.939712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:55.940062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:55.941424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:55.953086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 619/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:55.958958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.056868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fac617-9025-4aaa-921e-ade991f240b6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.089529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.090353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.090638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.090915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.098193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.102689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:00:56.113762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:56.114970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:56.116466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 607/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:56.121587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.227091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978f20-7b4a-4b8e-8918-be91bf8864fa None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.260217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.260875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.261159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.261436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.262292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:56.262608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.264038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.280528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 620/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:00:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:56.286282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.381552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd570e-a954-4ea5-91f8-f3e1bc17fd96 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.412485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.413255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.413533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.413803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.424959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.426765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:56.444243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:56.448310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:56.449228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:56.449228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:56.449228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:56.449228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:56.449228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 608/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:56.451228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.562454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded672b-5684-4ae0-aacb-34a56ecb18ad None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.596724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.597388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.597670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.597985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.598757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:56.599085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.600475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.611276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 621/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:00:56.617014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.718564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bea83-873d-4f81-ac91-96caecc7c8a6 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.751664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.752943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.753222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.753495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.760085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.761562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:00:56.775959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:56.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:00:56.778096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 609/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:56.783125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:56.897883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334bee-1c43-4661-92ef-25de4d577770 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:56.930168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:56.930878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:56.931160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:56.931433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:56.932212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:56.932524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:56.933947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5699a-34c9-4435-9288-6faba1318ece None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:56.947372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 622/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:56.952340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:57.046990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828ff4-7549-429d-bd11-713b0e28a104 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:57.079992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142120-e718-449f-b931-60ba512f3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:57.080933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142120-e718-449f-b931-60ba512f3048 None admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:57.081388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142120-e718-449f-b931-60ba512f3048 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:57.081671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142120-e718-449f-b931-60ba512f3048 None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/639566a1a9de49ef841486e9e21a2116/roles/5137301ee6174827bfa3dc85509cf1c0` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:57.095631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da142120-e718-449f-b931-60ba512f3048 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=639566a1a9de49ef841486e9e21a2116, role_id=5137301ee6174827bfa3dc85509cf1c0)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:57.097363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da142120-e718-449f-b931-60ba512f3048 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:57.194723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 610/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 14:00:56 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/639566a1a9de49ef841486e9e21a2116/roles/5137301ee6174827bfa3dc85509cf1c0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:00:59.401131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b1c0-acd2-4789-bb1f-9ab31d222142 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:59.401689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b1c0-acd2-4789-bb1f-9ab31d222142 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:59.401689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b1c0-acd2-4789-bb1f-9ab31d222142 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:59.403074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 623/1233] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:00:59.408187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:59.408598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:59.409314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:59.426029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:00:59.427237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:00:59.490790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 611/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:00:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:00:59.515147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b1c0-acd2-4789-bb1f-9ab31d222142 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:59.614904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b1c0-acd2-4789-bb1f-9ab31d222142 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:59.647351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:59.648188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:59.648487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:59.648761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:59.658475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:00:59.666658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:59.667740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:00:59.668775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 624/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Tue May 10 14:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:59.673779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:59.770811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1c21d-15b9-41db-8748-582bb5bb94df None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:59.804738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:59.805397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:59.805677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:59.805948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:59.806712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] RBAC: Adding query filter params (name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:00:59.807018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:59.808376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:59.816462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 612/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Tue May 10 14:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:00:59.822475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:00:59.919929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94cde6-9ee1-45b8-b671-f457ea1fdacb None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:00:59.952067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:00:59.952930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:00:59.953211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:00:59.953488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:00:59.964617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:00:59.967043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:00:59.983720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:00:59.990792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:00:59.991751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:00:59.991751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:00:59.991751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:00:59.991751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:00:59.991751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 625/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:00:59.992088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:00.088399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee636304-e701-4b36-b29b-81225c88b46d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:00.120302 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:00.120963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:00.121236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:00.121518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:00.122242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:00.122565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:00.123895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:00.136023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 613/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:00.141783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:00.236348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad55864-3992-4139-a477-735c255d5c16 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:00.270138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:00.270970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:00.271250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:00.271523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:00.279927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:00.281423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:01:00.296457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:00.297546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:00.298760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 626/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:00.304589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:00.398211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f0e9-40ba-4290-ba74-b1ce098c813e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:00.431084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:00.431742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:00.432051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:00.432303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:00.433128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:00.433438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:00.434884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:00.451298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 614/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:01:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:00.457269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:00.563901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba1fec-4178-4d4f-b77c-4ef6f5fc2b38 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:00.601928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:00.602605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:00.602887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:00.603161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:00.668136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:00.669589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:00.769055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 627/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 14:01:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:01:00.775174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:00.880641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531005c5-65c0-48ad-ab42-d58077643552 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:00.915472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:00.916264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:00.916549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:00.916826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:00.926713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:00.928146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:00.945680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:00.946786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:00.947812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:00.947812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:00.947812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:00.947812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:00.947812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 615/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:00.952602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:01.052177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36b7ae-dc21-423c-b206-d40a4e021a47 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:01.086326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:01.086995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:01.087280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:01.087555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:01.088275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:01.088581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:01.090036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:01.101290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 628/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:01.106989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:01.206235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58c8cc-0d0d-4ce6-826f-a4b0b37597de None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:01.238304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:01.239040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:01.239321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:01.239598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:01.246218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:01.250750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:01:01.261787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:01.262869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:01.267097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 616/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:01.269957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:01.366603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2179c2-c100-4a17-b31a-b49b2f81614e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:01.400515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:01.401185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:01.401469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:01.401745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:01.402558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:01.402868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:01.404328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed834fa-494c-4516-8044-7dbeab897b99 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:01.416018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 629/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:01.422250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:01.518768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843367c-391c-46b1-8c67-aef915776aab None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:01.551539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:01.552190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:01.552473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:01.552753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:01.553568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] RBAC: Adding query filter params (role.id=5137301ee6174827bfa3dc85509cf1c0, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=639566a1a9de49ef841486e9e21a2116) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:01.553877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:01.555317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f7ef-6fd3-473f-8c74-6edf44f97e88 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:01.560971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 617/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 14:01:01 2022] GET /identity/v3/role_assignments?user.id=639566a1a9de49ef841486e9e21a2116&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=5137301ee6174827bfa3dc85509cf1c0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:03.879267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fc16-3ceb-424d-a31d-020421a4e8ba None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:03.879846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fc16-3ceb-424d-a31d-020421a4e8ba None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:03.879846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4fc16-3ceb-424d-a31d-020421a4e8ba None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:03.881166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 630/1247] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:01:03.886362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:03.891147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:03.891147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:03.904022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:03.905209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:03.970026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 618/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:03.994738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4fc16-3ceb-424d-a31d-020421a4e8ba None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.094810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4fc16-3ceb-424d-a31d-020421a4e8ba None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.129810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.130620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.130901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.131169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.140703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:01:04.148199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:04.149423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:04.150774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 631/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 14:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:04.155293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.253893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e496-94c5-4389-a247-ab78ddaf7461 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.286662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.287335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.287609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.287883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.288623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:04.288930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:04.290326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:04.298171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 619/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:04.304400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.401634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264237f-625d-43c9-9602-0fea82a1bb46 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.433772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.434584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.434862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.435137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.445560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:04.447035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:04.464026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:04.465110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:04.466391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:04.466391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:04.466391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:04.466391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:04.466391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 632/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:04.470990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.568760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5e979-863f-4189-aa3b-89e6daa7e583 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.603953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.604611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.604890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.605164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.605875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:04.606177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:04.607590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:04.619505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 620/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:04.625470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.723106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e8dad-2909-4192-9cab-e2487953234b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.755471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.756272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.756553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.756829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.764474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:04.766001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:01:04.781763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:04.783121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:04.785314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 633/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:04.789912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:04.886534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b04288-d86f-4651-8b57-2844b48d15c9 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:04.919192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:04.919848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:04.920126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:04.920397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:04.921215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:04.921517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:04.922941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:04.941353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 621/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:04.947106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.044775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27b7f2-88bd-45e6-a964-4168f6c3cd0d None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:05.077090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:05.077773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:05.078057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:05.078361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:05.144014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:05.145398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:05.239025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 634/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 14:01:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:01:05.244784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.339082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e560bc7-3c67-4e3a-a083-30b2d17b9b75 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:05.371104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:05.371883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:05.372161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:05.372432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:05.383261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:05.384689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:05.401438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:05.402497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:05.403461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:05.403461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:05.403461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:05.403461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:05.403461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 622/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:05.408265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.504991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf784f4-fb09-4335-85d5-3d834253898a None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:05.537560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:05.538232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:05.538516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:05.538788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:05.539502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:05.539808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:05.541160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:05.552583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 635/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:05.558108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.655213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5762e7c-6ecb-4bac-be91-52d9da03d85d None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:05.687976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:05.688756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:05.689038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:05.689311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:05.695857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:05.697332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:01:05.710884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:05.712081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:05.713011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 623/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:05.717923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.819249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a53d9-faab-404b-acde-5de1711278ab None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:05.852569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:05.853218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:05.853497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:05.853780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:05.854586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:05.854889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:05.856372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe16cf-da9e-46b4-8c6a-55afd6d6322f None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:05.870725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 636/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:05.877130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:05.974235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb555-8466-474f-99d3-4d35077195e2 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:06.006307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:06.006954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:06.007238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:06.007515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:06.008334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=639566a1a9de49ef841486e9e21a2116) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:06.008712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:06.010126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf35db-d3fd-4e21-b524-d8f1fdc6a5ff None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:06.015472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 624/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 14:01:05 2022] GET /identity/v3/role_assignments?user.id=639566a1a9de49ef841486e9e21a2116&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:08.215342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381142fa-f4a0-4544-abce-bf9442e405fa None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.215913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381142fa-f4a0-4544-abce-bf9442e405fa None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.215913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381142fa-f4a0-4544-abce-bf9442e405fa None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.217253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 637/1261] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:08.222427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.222887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.222982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.239905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:08.241084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:08.304469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 625/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:08.327120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381142fa-f4a0-4544-abce-bf9442e405fa None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:08.426375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381142fa-f4a0-4544-abce-bf9442e405fa None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:08.458788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:08.459607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.459885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.460158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.470322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:01:08.477631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:08.478829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:08.479809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 638/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 14:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:08.484986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:08.580685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6135-52e7-4319-a467-9f078796f00b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:08.614809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:08.615469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.615750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.616022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.616761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:08.617064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:08.618450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:08.626648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 626/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 14:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:08.639159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:08.730706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3dee5-2b8f-45e7-965e-daa16bd982d0 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:08.763991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:08.764780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.765092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.765341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.775526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:08.776965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:08.793890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:08.794980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:08.796078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:08.796078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:08.796078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:08.796078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:08.796078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 639/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:08.801762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:08.901248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21afcd0-e7cf-4b05-9e6a-cac226fe4cd2 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:08.934269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:08.934933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:08.935214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:08.935538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:08.936258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:08.936564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:08.937922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:08.949554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 627/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:08.955474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.053674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1690517-1090-4a40-8e67-4b6c7383627e None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.086051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.086870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.087149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.087418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.095535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.097770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:01:09.111859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:09.113132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:09.114503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 640/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:09.118897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.215423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a74154-713f-49b0-9855-9fabb65e4941 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.250863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.251793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.252082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.252363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.253543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:09.253925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.256012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.273550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 628/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:01:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:09.279286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.378822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0bc2e-7bc9-46b7-884c-c5d3b2163cd9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.412573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.413358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.413643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.413921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.424207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.425766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:09.443006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:09.444067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:09.446339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:09.446339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:09.446339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:09.446339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:09.446339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 641/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:01:09.449913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.545255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c643448-5bc3-4597-9e80-b8cb4724e108 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.578601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.579262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.579614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.579888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.581250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:09.581735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.583127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.594122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 629/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:09.600398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.698111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81e5dd-9884-46f2-97f0-650c96b865ab None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.731098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.731889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.732169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.732440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.738945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.743846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:01:09.754675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:09.755757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:09.756942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 642/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:09.761612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:09.858736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf486-13b0-4314-b355-e0e4c5533e4e None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:09.892732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:09.893534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:09.893951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:09.894255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:09.895037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:09.895346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:09.896765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686a13a-6919-449d-9831-c1e09ed73f66 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:09.909447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 630/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:09.915400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:10.013944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662d68-49ac-4ced-818f-f40c68fef169 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:10.046881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:10.047795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:10.048074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:10.048346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] PATH_INFO: `/v3/projects/866dc84015f94210b74f03308ecf7a96/users/639566a1a9de49ef841486e9e21a2116/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:10.051220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] RBAC: Authorizing `identity:create_grant(project_id=866dc84015f94210b74f03308ecf7a96, user_id=639566a1a9de49ef841486e9e21a2116, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:10.052645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8668f-375b-4a9d-ae26-460556b4d5ea None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:10.137451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 643/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Tue May 10 14:01:09 2022] PUT /identity/v3/projects/866dc84015f94210b74f03308ecf7a96/users/639566a1a9de49ef841486e9e21a2116/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:01:12.379387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3d1d-2e95-4502-ad9b-dd6f499df354 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:12.379954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3d1d-2e95-4502-ad9b-dd6f499df354 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:12.379954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3d1d-2e95-4502-ad9b-dd6f499df354 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:12.381304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 631/1274] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:12.386541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:12.386964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:12.387175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:12.404157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:12.405338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:12.469782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 644/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:12.493604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc3d1d-2e95-4502-ad9b-dd6f499df354 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:12.590338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc3d1d-2e95-4502-ad9b-dd6f499df354 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:12.625502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:12.626324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:12.626606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:12.626926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:12.637134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(role_id) May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 10 14:01:12.645443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:12.647023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:12.648092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 632/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Tue May 10 14:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:12.652645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:12.750700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124d61-8929-4b21-ac6d-fa97ee1f9f5e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:12.783745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:12.784995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:12.785276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:12.785549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:12.786398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:12.786701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:12.788055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:12.795748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 645/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Tue May 10 14:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:12.801867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:12.899866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccd7bd-d160-418f-b42f-033aced4f8b3 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:12.932284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:12.933063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:12.933344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:12.933615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:12.943833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:12.945284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:12.962263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:12.963292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:12.964492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:12.964492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:12.964492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:12.964492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:12.964492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 633/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:12.969352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:13.067353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470aa1fd-1c6e-49f2-be5e-2a921c307cff None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:13.100140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:13.100806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:13.101087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:13.101360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:13.102074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:13.102479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:13.103825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:13.115660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 646/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:13.121303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:13.219894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d649-b0ac-4c76-aa48-0e06b4cf85d7 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:13.251744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:13.253102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:13.253411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:13.253679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:13.260826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:13.262275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_user(user_id) May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 10 14:01:13.276778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:13.277885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:13.279157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 634/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:13.283706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:13.380509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9719a-e27a-4ad4-b89f-07fa9d53398b None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:13.414071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:13.414764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:13.415085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:13.415357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:13.416195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:13.416485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:13.417890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:13.435259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 647/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Tue May 10 14:01:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:13.442139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:13.540780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978c11-c0d0-4fd2-abd3-b54a7f9c9fc0 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:13.573063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:13.573740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:13.574025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:13.574330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:13.651053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:13.653397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:13.770297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 635/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Tue May 10 14:01:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 10 14:01:13.776401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:13.881060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bfee0-8391-4e71-a426-3c8aa44eb938 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:13.917681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:13.918570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:13.918853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:13.919127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:13.930016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:13.931570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 10 14:01:13.950920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:13.952107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:13.953150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 10 14:01:13.953150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 10 14:01:13.953150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 10 14:01:13.953150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:13.953150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 648/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 10 14:01:13.958406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:14.064131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a204feb-42a0-462f-a80c-af11f6e133e5 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:14.098599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:14.099303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:14.099587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:14.099861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:14.100586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:14.100887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:14.102296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:14.114894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 636/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Tue May 10 14:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:14.120580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:14.219021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef420-54c0-4e10-a690-a58e9f2728ed None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:14.251863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:14.252765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:14.253050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:14.253328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:14.259898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:14.264732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(project_id) May 10 14:01:14.275517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application with Lock( May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:14.276568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: ERROR keystone.server.flask.application May 10 14:01:14.277682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 649/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Tue May 10 14:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:14.282570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:14.385242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49908754-8d9c-407d-a503-3b74b417871a None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:14.417740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:14.418419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:14.418702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:14.418977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:14.419824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:14.420133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:14.421545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828dbe-449a-4442-bf8e-2af6c0e37bc9 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:14.433015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 637/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Tue May 10 14:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:14.439084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:14.535474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061327af-92a2-4f48-86d8-814166ebb3f1 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:14.567531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:14.568229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:14.568513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:14.568793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:14.569797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] RBAC: Adding query filter params (role.id=1c3f38f1141a421ea1f2f6dd9bff328f, scope.project.id=866dc84015f94210b74f03308ecf7a96, user.id=639566a1a9de49ef841486e9e21a2116) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:14.570113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:14.571589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8a877-93c3-4fd3-b5fc-5a2196466658 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:14.577307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 650/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Tue May 10 14:01:14 2022] GET /identity/v3/role_assignments?user.id=639566a1a9de49ef841486e9e21a2116&scope.project.id=866dc84015f94210b74f03308ecf7a96&role.id=1c3f38f1141a421ea1f2f6dd9bff328f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:16.836789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535dca-e605-45ea-8140-f14eef83e6da None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:16.837403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535dca-e605-45ea-8140-f14eef83e6da None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:16.837403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535dca-e605-45ea-8140-f14eef83e6da None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:16.838801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 638/1288] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:16.844068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:16.844470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:16.855974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:16.860978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:16.862370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:16.925783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 651/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:16.949167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd535dca-e605-45ea-8140-f14eef83e6da None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:17.047994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd535dca-e605-45ea-8140-f14eef83e6da None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:17.080367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:17.082186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:17.082495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:17.086606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:17.086606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:17.086606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:17.096779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:17.098011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:17.099195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 639/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 14:01:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:17.104403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:17.200593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1f25-c84f-4b28-84ed-08f162291e21 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:17.232598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:17.233289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:17.233569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:17.233842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:17.234567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:17.234869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:17.242047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae5afb-c3b8-44f8-965e-49517688ee68 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:17.242047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 652/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 10 14:01:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:17.247265 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:17.343270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8d8a7-c213-4352-95d7-b18b59f4d621 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:17.379597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:17.380270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:17.380555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:17.380831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] PATH_INFO: `/v3/services` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:17.381537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:17.381893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:17.383299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10b36-79fa-4831-9b3d-2cf7f52016c7 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:17.387751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 640/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 10 14:01:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:19.571334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e330b5-baf9-4104-ae73-784a31ac6577 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:19.571936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e330b5-baf9-4104-ae73-784a31ac6577 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:19.571936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e330b5-baf9-4104-ae73-784a31ac6577 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:19.573381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 653/1293] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:19.578716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5cd02-e27c-4f3a-b929-ea15f6b6416a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:19.579140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5cd02-e27c-4f3a-b929-ea15f6b6416a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:19.579314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5cd02-e27c-4f3a-b929-ea15f6b6416a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:19.595553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7cd5cd02-e27c-4f3a-b929-ea15f6b6416a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:19.596782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7cd5cd02-e27c-4f3a-b929-ea15f6b6416a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:19.661171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 641/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:19.687933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e330b5-baf9-4104-ae73-784a31ac6577 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:19.798638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e330b5-baf9-4104-ae73-784a31ac6577 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:19.832059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:19.832675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:19.832982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:19.833256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:19.833975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:19.835374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a76bf-6b82-474f-98ec-5656d28e75d3 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:19.908061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 654/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Tue May 10 14:01:19 2022] POST /identity/v3/services => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:01:22.132599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c55844-f80c-4ced-b6f7-9f550944506c None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.133120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c55844-f80c-4ced-b6f7-9f550944506c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.133213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c55844-f80c-4ced-b6f7-9f550944506c None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.134542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 642/1296] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:22.139690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.140099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.140252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.157631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:22.159011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:22.228451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 655/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:22.251022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c55844-f80c-4ced-b6f7-9f550944506c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:22.348616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c55844-f80c-4ced-b6f7-9f550944506c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:22.386046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:22.386903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.387189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.387465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.388118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:22.389484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:22.402523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:22.403759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:22.404812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 643/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 14:01:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:22.410694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:22.508339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e0240-7bf4-4d00-9c17-55d9ecca7c1b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:22.541255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:22.541913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.542223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.542599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.543303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:22.543611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:22.544981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:22.550970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 656/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 10 14:01:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:22.557056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:22.654496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbedd9-33fb-4b0f-8966-d899ed11d318 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:22.686099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:22.686789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.687074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.687350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.688211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] RBAC: Adding query filter params (interface=public, service_id=c54233124e3649b1970137a149db2572, region_id=RegionOne) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:22.688522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:22.689875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be8cd-0efe-4e70-b6cf-f86419d6d8df None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:22.695520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 644/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Tue May 10 14:01:22 2022] GET /identity/v3/endpoints?service_id=c54233124e3649b1970137a149db2572&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:22.701328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:22.802848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e1c3-ea44-4624-9ad8-b4e5156610a9 None admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:22.835663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:22.836318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:22.836601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:22.836876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:22.837550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:22.838982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce593499-cd3e-45ad-b654-444f195633b6 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:22.845528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 657/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Tue May 10 14:01:22 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:01:25.210604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780937-cc9a-42fe-8b42-fcc58a51c8bb None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:25.211170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780937-cc9a-42fe-8b42-fcc58a51c8bb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:25.211170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb780937-cc9a-42fe-8b42-fcc58a51c8bb None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:25.212576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 645/1302] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:01:25.218002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:25.218436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:25.218621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:25.235132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:01:25.236312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:01:25.301282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 658/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:01:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 10 14:01:25.324192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb780937-cc9a-42fe-8b42-fcc58a51c8bb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:25.427148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb780937-cc9a-42fe-8b42-fcc58a51c8bb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:25.459649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:25.460432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:25.460760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:25.460983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:25.461672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:25.463040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application Traceback (most recent call last): May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 10 14:01:25.475604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._get_service(service_id) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 10 14:01:25.476743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.get_or_create( May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application with Lock( May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self._enter() May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application return self.creator() May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application created_value = creator( May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: ERROR keystone.server.flask.application May 10 14:01:25.477580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 646/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Tue May 10 14:01:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 10 14:01:25.483123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:25.577356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e88c9-41df-4961-8f4c-43a3176f3341 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:25.609832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:25.610517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:25.610801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:25.611084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] PATH_INFO: `/v3/services` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:25.611779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:01:25.612214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:25.613575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fea488-7ed6-4418-bd26-d3514d9287c9 None admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:25.618680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 659/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Tue May 10 14:01:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:01:25.624509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:01:25.721416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783815e-8cec-4cbc-802f-05ac338a6504 None admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:01:25.754507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:01:25.755165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:01:25.755448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:01:25.755727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:01:25.756466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:01:25.757822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d6134-cc26-4264-b540-8b09ac7ed310 None admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:01:25.823152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 647/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Tue May 10 14:01:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:07:37.520781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1f53-68f6-40fe-aa96-f8f87b146496 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:37.521804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1f53-68f6-40fe-aa96-f8f87b146496 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:37.522337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1f53-68f6-40fe-aa96-f8f87b146496 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:37.529746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6dc43-1930-414b-82d1-10c7d79e357a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:37.533399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6dc43-1930-414b-82d1-10c7d79e357a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:37.533399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6dc43-1930-414b-82d1-10c7d79e357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:37.533606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 660/1307] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:37.624213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-91e6dc43-1930-414b-82d1-10c7d79e357a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:37.625795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-91e6dc43-1930-414b-82d1-10c7d79e357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:37.846129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 648/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:37.871980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:37.872240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:37.872512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:37.874151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 661/1309] 10.222.0.58 () {54 vars in 964 bytes} [Tue May 10 14:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:37.880201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c6af-57d0-4dfc-8fca-4de3b8462953 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:37.880498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c6af-57d0-4dfc-8fca-4de3b8462953 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:37.880782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c6af-57d0-4dfc-8fca-4de3b8462953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:37.968765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cfd3c6af-57d0-4dfc-8fca-4de3b8462953 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:37.969974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cfd3c6af-57d0-4dfc-8fca-4de3b8462953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:38.052728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 649/1310] 10.222.0.58 () {58 vars in 1061 bytes} [Tue May 10 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:38.062068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:38.064231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:38.184299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d12e18-bd17-424b-a058-29b1b6813616 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:38.260727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:38.262149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:38.262478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:38.262760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:38.264431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:38.380514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:38.381962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033e1da-5f82-40e7-a8d8-fab56082cc02 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:38.406261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 662/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:07:41.025429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02da7c-879b-4812-8821-b44d36a8a4ef None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:41.025967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02da7c-879b-4812-8821-b44d36a8a4ef None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:41.025967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02da7c-879b-4812-8821-b44d36a8a4ef None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:41.028134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 650/1312] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:41.034231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ec85-ea64-4cfd-b0cb-e1483a164ab0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:41.034766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ec85-ea64-4cfd-b0cb-e1483a164ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:41.034850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ec85-ea64-4cfd-b0cb-e1483a164ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:41.135330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-9665ec85-ea64-4cfd-b0cb-e1483a164ab0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:41.137211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9665ec85-ea64-4cfd-b0cb-e1483a164ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:41.249277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 663/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:41.304592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:41.304978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:41.305330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:41.306854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 651/1314] 10.222.0.58 () {54 vars in 964 bytes} [Tue May 10 14:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:41.314261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9b67-dff6-4318-b173-a0f91ffce5f1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:41.314701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9b67-dff6-4318-b173-a0f91ffce5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:41.315050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9b67-dff6-4318-b173-a0f91ffce5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:41.390544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-46fd9b67-dff6-4318-b173-a0f91ffce5f1 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:41.391844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-46fd9b67-dff6-4318-b173-a0f91ffce5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:41.468964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 664/1315] 10.222.0.58 () {58 vars in 1061 bytes} [Tue May 10 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:41.477729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:41.479272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:41.588777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa9ff-2133-44b5-b66f-2a1f389c4968 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:41.666170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:41.666853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:41.667167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:41.667534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:41.669190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:41.788991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:41.790618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c72d6-ef6e-4776-afe0-7df1748972e4 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:41.809822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 652/1316] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:07:44.826520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:44.827110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:44.827110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:44.828468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 665/1317] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:44.833619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead2bf-64a7-4bb5-b263-25f588b8772c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:44.834048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead2bf-64a7-4bb5-b263-25f588b8772c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:44.834175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead2bf-64a7-4bb5-b263-25f588b8772c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:44.896476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-baead2bf-64a7-4bb5-b263-25f588b8772c None None] MFA Rules not processed for user `deac13a59faa44bab73388e308e1cb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:44.897664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-baead2bf-64a7-4bb5-b263-25f588b8772c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:44.989920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 653/1318] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:45.030523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:45.033772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:45.140616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9d36f-6ba7-4a75-9738-66bb5f656dc4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:45.221709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:45.222397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:45.222686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:45.222964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:45.224623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:45.330397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:45.331744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f10e01-dd83-4739-9637-92267cbc3ea3 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:45.349076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 666/1319] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:07:48.365156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:48.365698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:48.365698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:48.367094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 654/1320] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:48.373165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311794d5-4e94-4449-a20c-09e778f9fb08 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:48.373625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311794d5-4e94-4449-a20c-09e778f9fb08 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:48.373734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311794d5-4e94-4449-a20c-09e778f9fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:48.435885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-311794d5-4e94-4449-a20c-09e778f9fb08 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:48.437091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-311794d5-4e94-4449-a20c-09e778f9fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:48.530387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 667/1321] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:48.570829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:48.572372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:48.672980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c0069-facb-49de-b827-efba7ab86aa3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:48.742782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:48.743426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:48.743705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:48.743976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:48.746374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:48.849445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:48.850919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d9a7-3545-4d71-a656-d77525dcf1fd service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:48.868651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 655/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:07:52.288475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:52.289094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:52.289094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:52.290421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 668/1323] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:52.296494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e549-e438-4b8a-b54c-e7c4975de165 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:52.296954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e549-e438-4b8a-b54c-e7c4975de165 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:52.297068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e549-e438-4b8a-b54c-e7c4975de165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:52.362374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ffe5e549-e438-4b8a-b54c-e7c4975de165 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:52.363574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ffe5e549-e438-4b8a-b54c-e7c4975de165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:52.456924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 656/1324] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:52.492017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:52.493696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:52.595436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836dcfb-9098-4002-82c4-98f9aed05343 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:52.671957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:52.672607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:52.672890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:52.673162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:52.674827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:52.783813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:52.785250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab540e5-5b2c-43f3-bcf6-b8c3642deb78 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:52.803906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 669/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:07:55.127756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:55.128276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:55.128366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:55.129695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 657/1326] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:55.135290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ad8a-933b-4e6a-86e8-b30b15f70822 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:55.135711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ad8a-933b-4e6a-86e8-b30b15f70822 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:55.135845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ad8a-933b-4e6a-86e8-b30b15f70822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:55.200249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b320ad8a-933b-4e6a-86e8-b30b15f70822 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:55.201458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b320ad8a-933b-4e6a-86e8-b30b15f70822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:55.306185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 670/1327] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:55.343285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:55.344799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:55.451147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb56c2-50ed-42ed-bd52-2716e83fc9ff None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:55.521053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:55.521698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:55.521978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:55.522280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:55.523875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:55.628770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:55.630182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e145a88-ff92-469c-b10f-4399d7c35b9f service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:55.648819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 658/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:07:57.940278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:57.940782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:57.940782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:57.942162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 671/1329] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:07:57.947687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883404-efdc-444b-8268-e00896957b8f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:57.948134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883404-efdc-444b-8268-e00896957b8f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:57.948233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53883404-efdc-444b-8268-e00896957b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:58.009941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-53883404-efdc-444b-8268-e00896957b8f None None] MFA Rules not processed for user `deac13a59faa44bab73388e308e1cb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:07:58.011141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-53883404-efdc-444b-8268-e00896957b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:58.103792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 659/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:07:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:07:58.141183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:07:58.142776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:58.250291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e89488-7b7a-4634-9f78-0222b4e702a7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:07:58.324844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:07:58.325496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:07:58.325777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:07:58.326049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:07:58.327720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:07:58.440190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:07:58.441669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6d38a-15bf-4615-bbc6-8f945fac0ca3 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:07:58.463266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 672/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:07:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:01.178436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:01.182148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:01.182148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:01.186048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 660/1332] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:01.187332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f340c-27bd-4155-b531-f300ddfb03e8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:01.187772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f340c-27bd-4155-b531-f300ddfb03e8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:01.187885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f340c-27bd-4155-b531-f300ddfb03e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:01.259519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a58f340c-27bd-4155-b531-f300ddfb03e8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:01.260911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a58f340c-27bd-4155-b531-f300ddfb03e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:01.358658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 673/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:08:01.395641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:01.397159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:01.500928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb20ec-e6ac-4c66-acf4-50ce575a38ae None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:01.572388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:01.573041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:01.573316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:01.573584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:01.575221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:01.679297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:01.680722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e053d-5b06-4e17-84cb-4941363fc845 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:01.705994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 661/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:04.020902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:04.021434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:04.021434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:04.022778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 674/1335] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:04.028034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984735f-f68e-4141-8130-2757e2f006b2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:04.028483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984735f-f68e-4141-8130-2757e2f006b2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:04.028590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984735f-f68e-4141-8130-2757e2f006b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:04.091600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1984735f-f68e-4141-8130-2757e2f006b2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:04.092815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1984735f-f68e-4141-8130-2757e2f006b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:04.186569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 662/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:04.225809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:04.227387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:04.326071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679a4e6-dbc5-4f18-9727-17240c1ecdcc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:04.395941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:04.396593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:04.396876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:04.397151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:04.398796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:04.505371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:04.506887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e5dea-cd5d-41b3-8623-ca8d2bfc9476 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:04.525139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 675/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:07.599279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:07.599854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:07.599854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:07.601201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 663/1338] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:07.606736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79aeb8-9831-48e5-b436-b5a68aaa6551 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:07.607165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79aeb8-9831-48e5-b436-b5a68aaa6551 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:07.607287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79aeb8-9831-48e5-b436-b5a68aaa6551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:07.671673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7f79aeb8-9831-48e5-b436-b5a68aaa6551 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:07.672888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7f79aeb8-9831-48e5-b436-b5a68aaa6551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:07.766370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 676/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:07.802299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:07.805786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:07.914157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a778cd-2dea-4385-8d38-53127eae2acc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:07.918384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:07.919033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:07.919318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:07.919626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:07.921392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:08.022959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:08.025117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132b028-85e1-4367-80fe-a59b430f8be5 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:08.044173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 664/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:12.573993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bca2c9-ffdf-4310-bcb8-e6702c8dd2fd None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:12.574573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bca2c9-ffdf-4310-bcb8-e6702c8dd2fd None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:12.574573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bca2c9-ffdf-4310-bcb8-e6702c8dd2fd None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:12.576007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 677/1341] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:12.581784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52e38-0a59-40d2-9c88-3ce470eba487 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:12.582271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52e38-0a59-40d2-9c88-3ce470eba487 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:12.582393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52e38-0a59-40d2-9c88-3ce470eba487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:12.629410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fcf52e38-0a59-40d2-9c88-3ce470eba487 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:12.630645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fcf52e38-0a59-40d2-9c88-3ce470eba487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:12.706962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 665/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:08:12.747112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bca2c9-ffdf-4310-bcb8-e6702c8dd2fd None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:12.759313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bca2c9-ffdf-4310-bcb8-e6702c8dd2fd None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:12.763206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:12.763846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:12.764398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:12.764672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:12.774066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:12.775788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfe216-867a-4766-ab73-9454096a95b3 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:12.785491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 678/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:16.011691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de57-5b0b-44f4-8377-ae709e3eb493 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:16.012254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de57-5b0b-44f4-8377-ae709e3eb493 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:16.012254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de57-5b0b-44f4-8377-ae709e3eb493 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:16.013521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 666/1344] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:16.019257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3104ce-fbaa-4f1f-ab0f-1530dce14a1d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:16.019705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3104ce-fbaa-4f1f-ab0f-1530dce14a1d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:16.019965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3104ce-fbaa-4f1f-ab0f-1530dce14a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:16.047656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3a3104ce-fbaa-4f1f-ab0f-1530dce14a1d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:16.048830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3a3104ce-fbaa-4f1f-ab0f-1530dce14a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:16.057133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 679/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:16.094607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749de57-5b0b-44f4-8377-ae709e3eb493 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:16.105427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749de57-5b0b-44f4-8377-ae709e3eb493 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:16.109009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:16.109644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:16.109928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:16.110227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:16.119424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:16.120962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78cdf5-fdf0-4736-a233-e0983477ff08 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:16.130160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 667/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:20.802379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51281184-07b7-41d0-82f7-9ef120d3fca5 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:20.802941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51281184-07b7-41d0-82f7-9ef120d3fca5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:20.802941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51281184-07b7-41d0-82f7-9ef120d3fca5 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:20.804339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 680/1347] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:20.810170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd3a0e-94f6-461a-9364-de5094007ced None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:20.810609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd3a0e-94f6-461a-9364-de5094007ced None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:20.810777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd3a0e-94f6-461a-9364-de5094007ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:20.829871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6acd3a0e-94f6-461a-9364-de5094007ced None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:20.831098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6acd3a0e-94f6-461a-9364-de5094007ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:20.838997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 668/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:08:20.876352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51281184-07b7-41d0-82f7-9ef120d3fca5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:20.887660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51281184-07b7-41d0-82f7-9ef120d3fca5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:20.891410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:20.892040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:20.892321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:20.892597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:20.901398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:20.902845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2bf9b-eb51-4797-a6e0-6709e62a6722 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:20.912189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 681/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:26.391726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d435f-b6fe-454f-b528-2cb34cd314b4 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:26.392360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d435f-b6fe-454f-b528-2cb34cd314b4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:26.392360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d435f-b6fe-454f-b528-2cb34cd314b4 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:26.393672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 669/1350] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:26.400354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a640cbd-1fef-4bfd-817a-e6292c6b2bf2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:26.400792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a640cbd-1fef-4bfd-817a-e6292c6b2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:26.400886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a640cbd-1fef-4bfd-817a-e6292c6b2bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:26.473106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3a640cbd-1fef-4bfd-817a-e6292c6b2bf2 None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:26.474764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3a640cbd-1fef-4bfd-817a-e6292c6b2bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:26.556816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 682/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:30.921105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9fc8a-deec-4719-bbe6-b9fd5d5111dd None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:30.921705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9fc8a-deec-4719-bbe6-b9fd5d5111dd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:30.921705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9fc8a-deec-4719-bbe6-b9fd5d5111dd None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:30.923439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 670/1352] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:30.929514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6d23-38af-47e5-bca4-c4cac7c10c10 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:30.930087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6d23-38af-47e5-bca4-c4cac7c10c10 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:30.930426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6d23-38af-47e5-bca4-c4cac7c10c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:30.950980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3a3a6d23-38af-47e5-bca4-c4cac7c10c10 None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:30.952625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3a3a6d23-38af-47e5-bca4-c4cac7c10c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:30.961998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 683/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:32.570056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213afdb5-a2db-4fef-a0fc-399cabd70fea None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:32.570611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213afdb5-a2db-4fef-a0fc-399cabd70fea None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:32.570741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213afdb5-a2db-4fef-a0fc-399cabd70fea None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:32.572122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 671/1354] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:32.577307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b37236-9d6e-4176-bf33-d78abaf6d44e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:32.577735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b37236-9d6e-4176-bf33-d78abaf6d44e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:32.577858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b37236-9d6e-4176-bf33-d78abaf6d44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:32.604325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-78b37236-9d6e-4176-bf33-d78abaf6d44e None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:32.605739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-78b37236-9d6e-4176-bf33-d78abaf6d44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:32.614775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 684/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:08:36.546666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ed50-3505-4205-8fd8-ffea7f80c868 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:36.547208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ed50-3505-4205-8fd8-ffea7f80c868 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:36.547208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ed50-3505-4205-8fd8-ffea7f80c868 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:36.548552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 672/1356] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:36.553965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325574eb-8878-411e-a4bc-08aef9e798fb None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:36.554444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325574eb-8878-411e-a4bc-08aef9e798fb None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:36.554555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325574eb-8878-411e-a4bc-08aef9e798fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:36.571968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-325574eb-8878-411e-a4bc-08aef9e798fb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:36.575147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-325574eb-8878-411e-a4bc-08aef9e798fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:36.581396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 685/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:36.629116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e999-4e86-4798-8691-e6457e24e1ee None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:36.629689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e999-4e86-4798-8691-e6457e24e1ee None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:36.630164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e999-4e86-4798-8691-e6457e24e1ee None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:36.631666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 673/1358] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 10 14:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:36.637411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e5780-d5d0-46f0-ac3b-b3a111463f8c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:36.637774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e5780-d5d0-46f0-ac3b-b3a111463f8c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:36.638124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e5780-d5d0-46f0-ac3b-b3a111463f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:36.687065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-827e5780-d5d0-46f0-ac3b-b3a111463f8c None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:36.688388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-827e5780-d5d0-46f0-ac3b-b3a111463f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:36.784335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 686/1359] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 10 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:36.793480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e999-4e86-4798-8691-e6457e24e1ee None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:36.808393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e999-4e86-4798-8691-e6457e24e1ee None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:36.814157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:36.814932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:36.815458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:36.815824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:36.824505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:36.826220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a87c9-b584-4ce7-8f18-33da6a3220e9 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:36.836654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 674/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:39.307989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d9b-d74f-4226-8bd5-d08544623faa None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:39.308543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d9b-d74f-4226-8bd5-d08544623faa None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:39.308543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d9b-d74f-4226-8bd5-d08544623faa None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:39.312876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 687/1361] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:39.315322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8caa-814e-4ae9-a8b1-0c51a5eddb23 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:39.315759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8caa-814e-4ae9-a8b1-0c51a5eddb23 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:39.315888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8caa-814e-4ae9-a8b1-0c51a5eddb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:39.333770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d70a8caa-814e-4ae9-a8b1-0c51a5eddb23 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:39.335172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d70a8caa-814e-4ae9-a8b1-0c51a5eddb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:39.343594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 675/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:39.387877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3d9b-d74f-4226-8bd5-d08544623faa None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:39.403334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3d9b-d74f-4226-8bd5-d08544623faa None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:39.409293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:39.410017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:39.413373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:39.414133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:39.425343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:39.427290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f83a8-f0a4-402e-a26f-9f92190cfed3 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:39.438916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 688/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:42.144465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b2180-cff5-4818-bebe-e6e2f8f43e0a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:42.144979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b2180-cff5-4818-bebe-e6e2f8f43e0a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:42.144979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b2180-cff5-4818-bebe-e6e2f8f43e0a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:42.146358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 676/1364] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:42.152556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4d5-c5c5-45a8-ab90-326e125e50b4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:42.152984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4d5-c5c5-45a8-ab90-326e125e50b4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:42.153120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4d5-c5c5-45a8-ab90-326e125e50b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:42.172553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6cd0d4d5-c5c5-45a8-ab90-326e125e50b4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:42.173767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6cd0d4d5-c5c5-45a8-ab90-326e125e50b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:42.182118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 689/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:42.232576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91b9cb-f666-46f0-ac6a-b8113619011d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:42.232943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91b9cb-f666-46f0-ac6a-b8113619011d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:42.233330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91b9cb-f666-46f0-ac6a-b8113619011d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:42.234966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 677/1366] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 10 14:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:42.243168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072792-8259-4c10-91a4-53daf30c781d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:42.243529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072792-8259-4c10-91a4-53daf30c781d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:42.243899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072792-8259-4c10-91a4-53daf30c781d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:42.262276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-62072792-8259-4c10-91a4-53daf30c781d None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:42.263730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-62072792-8259-4c10-91a4-53daf30c781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:42.273689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 690/1367] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 10 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:42.281736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91b9cb-f666-46f0-ac6a-b8113619011d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:42.296375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91b9cb-f666-46f0-ac6a-b8113619011d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:42.301886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:42.302567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:42.302823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:42.303228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:42.311346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:42.312922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914f3f2-40e4-47d7-b4d8-de11ef3d5cab service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:42.323027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 678/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:44.856871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8d78c-d881-4b11-a0d1-c2acd43138ec None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:44.857471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8d78c-d881-4b11-a0d1-c2acd43138ec None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:44.857471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8d78c-d881-4b11-a0d1-c2acd43138ec None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:44.859118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 691/1369] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:44.864301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d5d1-1a33-4442-b2ac-ceddb3cf47dd None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:44.864742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d5d1-1a33-4442-b2ac-ceddb3cf47dd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:44.864860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6d5d1-1a33-4442-b2ac-ceddb3cf47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:44.884946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cdd6d5d1-1a33-4442-b2ac-ceddb3cf47dd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:44.886457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cdd6d5d1-1a33-4442-b2ac-ceddb3cf47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:44.895560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 679/1370] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:44.916723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcec0-0531-43ea-b39c-f480552abcee None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:44.917299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcec0-0531-43ea-b39c-f480552abcee None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:44.917766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcec0-0531-43ea-b39c-f480552abcee None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:44.919249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 692/1371] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:44.925235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304473d8-1951-4bfc-8297-430b65f9c364 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:44.925523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304473d8-1951-4bfc-8297-430b65f9c364 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:44.926053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304473d8-1951-4bfc-8297-430b65f9c364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:44.942241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bcec0-0531-43ea-b39c-f480552abcee None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:44.945103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-304473d8-1951-4bfc-8297-430b65f9c364 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:44.946392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-304473d8-1951-4bfc-8297-430b65f9c364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:44.954685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 680/1372] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:44.957599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bcec0-0531-43ea-b39c-f480552abcee None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:44.963856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:44.964603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:44.964952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:44.965316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:44.973909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:44.976305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c033e8-c00b-4b99-b6ee-d3ffd2e799da service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:44.986461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 693/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:45.276969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79da6cb-cb82-42a0-ae96-c3b0f1340a91 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:45.276969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79da6cb-cb82-42a0-ae96-c3b0f1340a91 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:45.276969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79da6cb-cb82-42a0-ae96-c3b0f1340a91 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:45.280213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 681/1374] 10.222.0.58 () {54 vars in 963 bytes} [Tue May 10 14:08:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:45.290509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5c7f3-ccf8-44f7-a4c0-bc7a67a0934b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:45.290509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5c7f3-ccf8-44f7-a4c0-bc7a67a0934b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:45.290509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5c7f3-ccf8-44f7-a4c0-bc7a67a0934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:45.345220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a8c5c7f3-ccf8-44f7-a4c0-bc7a67a0934b None None] MFA Rules not processed for user `0ef044e8d1fb4953b2cc3244f4a4369f`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:45.346737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a8c5c7f3-ccf8-44f7-a4c0-bc7a67a0934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:45.409084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 694/1375] 10.222.0.58 () {58 vars in 1060 bytes} [Tue May 10 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:45.417196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79da6cb-cb82-42a0-ae96-c3b0f1340a91 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:45.428300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79da6cb-cb82-42a0-ae96-c3b0f1340a91 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:45.432099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:45.433014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:45.433368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:45.433713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:45.443083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:45.445222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd930f-31d4-46ad-960a-0b5fda7f54eb service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:45.455785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 682/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:08:47.373025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73986d-3fc9-4c90-b749-f8c3c6d3ee4c None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:47.373552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73986d-3fc9-4c90-b749-f8c3c6d3ee4c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:47.373552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73986d-3fc9-4c90-b749-f8c3c6d3ee4c None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:47.374896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 695/1377] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:47.380289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608812e-cba6-4441-a4ab-8dfaecf1a55e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:47.380783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608812e-cba6-4441-a4ab-8dfaecf1a55e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:47.381033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608812e-cba6-4441-a4ab-8dfaecf1a55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:47.399971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3608812e-cba6-4441-a4ab-8dfaecf1a55e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:47.401220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3608812e-cba6-4441-a4ab-8dfaecf1a55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:47.409180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 683/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:47.453395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73986d-3fc9-4c90-b749-f8c3c6d3ee4c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:47.467727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73986d-3fc9-4c90-b749-f8c3c6d3ee4c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:47.473342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:47.473975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:47.474298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:47.475112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:47.483645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:47.485247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190e6ea-cda0-4da6-851f-76ea88ea5e8f service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:47.495550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 696/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:49.941143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe300a9-0c04-4eb2-b108-4cd2f95db237 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:49.941723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe300a9-0c04-4eb2-b108-4cd2f95db237 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:49.941723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe300a9-0c04-4eb2-b108-4cd2f95db237 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:49.943610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 684/1380] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:49.949261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4bdc6-cd5b-4485-aadd-9bfeb585cf64 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:49.949718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4bdc6-cd5b-4485-aadd-9bfeb585cf64 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:49.949817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4bdc6-cd5b-4485-aadd-9bfeb585cf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:49.967728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-64f4bdc6-cd5b-4485-aadd-9bfeb585cf64 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:49.968889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-64f4bdc6-cd5b-4485-aadd-9bfeb585cf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:49.977281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 697/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:50.021010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe300a9-0c04-4eb2-b108-4cd2f95db237 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:50.036507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe300a9-0c04-4eb2-b108-4cd2f95db237 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:50.042872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:50.043617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:50.044025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:50.044403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:50.053768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:50.055591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5d173-2a91-449e-b19d-163ed6f60ebf service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:50.066308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 685/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:08:52.500740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6510b4-55d7-4416-a8e7-ac19313eea17 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:52.501326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6510b4-55d7-4416-a8e7-ac19313eea17 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:52.501326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6510b4-55d7-4416-a8e7-ac19313eea17 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:52.502695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 698/1383] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:52.508230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cfab-51f7-4b7d-9164-d16b7d12f3f1 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:52.508665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cfab-51f7-4b7d-9164-d16b7d12f3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:52.508792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84cfab-51f7-4b7d-9164-d16b7d12f3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:52.526978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ed84cfab-51f7-4b7d-9164-d16b7d12f3f1 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:52.528417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ed84cfab-51f7-4b7d-9164-d16b7d12f3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:52.536441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 686/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:08:52.580296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6510b4-55d7-4416-a8e7-ac19313eea17 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:52.595658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6510b4-55d7-4416-a8e7-ac19313eea17 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:52.601998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:52.602702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:52.602985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:52.603264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:52.611725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:52.613299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfee88-595b-4bbf-83ad-8f9aa5657488 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:52.623112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 699/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:54.882569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552318c6-c0a3-4351-9482-15038ccfb5f8 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:54.883263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552318c6-c0a3-4351-9482-15038ccfb5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:54.883263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552318c6-c0a3-4351-9482-15038ccfb5f8 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:54.884516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 687/1386] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:08:54.890228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9471c2-61af-4eaf-9200-fa9898b10ecc None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:54.890778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9471c2-61af-4eaf-9200-fa9898b10ecc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:54.890898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9471c2-61af-4eaf-9200-fa9898b10ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:54.908764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0d9471c2-61af-4eaf-9200-fa9898b10ecc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:54.910030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0d9471c2-61af-4eaf-9200-fa9898b10ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:54.919850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 700/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:54.965048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552318c6-c0a3-4351-9482-15038ccfb5f8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:54.979806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552318c6-c0a3-4351-9482-15038ccfb5f8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:54.985392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:54.986039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:54.986357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:54.986668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:54.995961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:54.997555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5341-428a-4768-beed-94c05b2e9c62 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:55.007640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 688/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:08:57.702676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebbcd-dfca-4d24-abae-ecc20e9fd692 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:57.703209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebbcd-dfca-4d24-abae-ecc20e9fd692 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:57.703314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebbcd-dfca-4d24-abae-ecc20e9fd692 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:57.704618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 701/1389] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:08:57.711110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0702-d1ba-45a9-bdc5-a9ce2aca52f6 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:57.712544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0702-d1ba-45a9-bdc5-a9ce2aca52f6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:57.712734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0702-d1ba-45a9-bdc5-a9ce2aca52f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:57.731439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-bead0702-d1ba-45a9-bdc5-a9ce2aca52f6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:08:57.732651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-bead0702-d1ba-45a9-bdc5-a9ce2aca52f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:08:57.741856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 689/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:08:57.787783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ebbcd-dfca-4d24-abae-ecc20e9fd692 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:08:57.802755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ebbcd-dfca-4d24-abae-ecc20e9fd692 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:08:57.809181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:08:57.810043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:08:57.810514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:08:57.810872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:08:57.821144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:08:57.822811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed56853-88c6-4a01-8592-0b73c4835033 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:08:57.832458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 702/1391] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:00.334382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997087-0b03-4ed0-84d7-1ebc6a320c3b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:00.335002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997087-0b03-4ed0-84d7-1ebc6a320c3b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:00.335191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997087-0b03-4ed0-84d7-1ebc6a320c3b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:00.336592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 690/1392] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:00.342314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad27a-7925-48f3-90f3-3c8d45035e69 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:00.342769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad27a-7925-48f3-90f3-3c8d45035e69 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:00.342878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad27a-7925-48f3-90f3-3c8d45035e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:00.361516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-874ad27a-7925-48f3-90f3-3c8d45035e69 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:00.363504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-874ad27a-7925-48f3-90f3-3c8d45035e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:00.370902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 703/1393] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:00.415960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2997087-0b03-4ed0-84d7-1ebc6a320c3b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:00.429910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2997087-0b03-4ed0-84d7-1ebc6a320c3b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:00.435378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:00.436013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:00.436294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:00.436567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:00.446139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:00.447784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1fc8-4545-4703-a22a-cb6c54305f36 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:00.457299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 691/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:01.471568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019478f-1921-4bba-933a-ef5618a70af2 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:01.472562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019478f-1921-4bba-933a-ef5618a70af2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:01.472879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019478f-1921-4bba-933a-ef5618a70af2 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:01.474352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 704/1395] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:01.480793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6f43-75ef-4992-94d6-34ed8cd0e7e5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:01.481280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6f43-75ef-4992-94d6-34ed8cd0e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:01.481540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b6f43-75ef-4992-94d6-34ed8cd0e7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:01.499775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e32b6f43-75ef-4992-94d6-34ed8cd0e7e5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:01.500963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e32b6f43-75ef-4992-94d6-34ed8cd0e7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:01.510390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 692/1396] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:01.566351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a885051-bf83-4db0-8249-199cc627d1a6 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:01.566647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a885051-bf83-4db0-8249-199cc627d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:01.566930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a885051-bf83-4db0-8249-199cc627d1a6 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:01.568342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 705/1397] 10.222.0.58 () {54 vars in 962 bytes} [Tue May 10 14:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:01.573409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9811-2842-472a-8df0-ca7a5206ef1b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:01.573699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9811-2842-472a-8df0-ca7a5206ef1b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:01.573980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9811-2842-472a-8df0-ca7a5206ef1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:01.621635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f05b9811-2842-472a-8df0-ca7a5206ef1b None None] MFA Rules not processed for user `4ee88945608d42c0981bb93470e832fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:01.622853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f05b9811-2842-472a-8df0-ca7a5206ef1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:01.681597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 693/1398] 10.222.0.58 () {58 vars in 1059 bytes} [Tue May 10 14:09:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:01.689118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a885051-bf83-4db0-8249-199cc627d1a6 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:01.700293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a885051-bf83-4db0-8249-199cc627d1a6 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:01.704706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:01.704706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:01.704958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:01.705132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:01.715038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:01.716474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85d420-3808-468d-ad36-7e95c9357007 service glance] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:01.726567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 706/1399] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:02.321649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848e7b-d4f8-4d07-a531-4feb950cfee3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.321939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848e7b-d4f8-4d07-a531-4feb950cfee3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.322234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848e7b-d4f8-4d07-a531-4feb950cfee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.370557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8d848e7b-d4f8-4d07-a531-4feb950cfee3 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:02.371737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d848e7b-d4f8-4d07-a531-4feb950cfee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:02.435629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 694/1400] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:02.484177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0251-de5b-4c75-9c9b-680fc06d048a None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.484681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0251-de5b-4c75-9c9b-680fc06d048a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.484809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0251-de5b-4c75-9c9b-680fc06d048a None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.486100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 707/1401] 10.222.0.58 () {54 vars in 942 bytes} [Tue May 10 14:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:02.493697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdff16f-ffe8-4b9d-aa84-64eb427b9d4f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.494166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdff16f-ffe8-4b9d-aa84-64eb427b9d4f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.494296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdff16f-ffe8-4b9d-aa84-64eb427b9d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.542028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-afdff16f-ffe8-4b9d-aa84-64eb427b9d4f None None] MFA Rules not processed for user `62ef235951c84f62b91e30d56860a904`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:02.543258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-afdff16f-ffe8-4b9d-aa84-64eb427b9d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:02.614260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 695/1402] 10.222.0.58 () {58 vars in 1039 bytes} [Tue May 10 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:02.624268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd0251-de5b-4c75-9c9b-680fc06d048a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:02.638492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd0251-de5b-4c75-9c9b-680fc06d048a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:02.643539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:02.644304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.644661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.645013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.654161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:02.655908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd3561-5c1e-4006-8abf-cfe22993a68b service swift] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:02.665238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 708/1403] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:09:02.886315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724ebe-91b4-423a-9381-06f28f6a2dde None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.886569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724ebe-91b4-423a-9381-06f28f6a2dde None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.886875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724ebe-91b4-423a-9381-06f28f6a2dde None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.888621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 696/1404] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:02.893968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2998d2-8c2c-40b5-a928-aa31488039c7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.894300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2998d2-8c2c-40b5-a928-aa31488039c7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.894587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2998d2-8c2c-40b5-a928-aa31488039c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.913497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4c2998d2-8c2c-40b5-a928-aa31488039c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:02.914704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4c2998d2-8c2c-40b5-a928-aa31488039c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:02.922796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 709/1405] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:02.967984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724ebe-91b4-423a-9381-06f28f6a2dde None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:02.982856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724ebe-91b4-423a-9381-06f28f6a2dde None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:02.988280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:02.988936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:02.989241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:02.989518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:02.997487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:02.999090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093e607-f516-42d3-bd3c-5c96467970e5 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:03.008488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 697/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:05.486837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f257-29d9-4023-8991-ea0d3d11d9c0 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:05.487367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f257-29d9-4023-8991-ea0d3d11d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:05.487597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f257-29d9-4023-8991-ea0d3d11d9c0 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:05.490093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 710/1407] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:05.495984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75656-3279-490d-b825-bb32f97cbb57 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:05.496806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75656-3279-490d-b825-bb32f97cbb57 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:05.497238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd75656-3279-490d-b825-bb32f97cbb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:05.519886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0cd75656-3279-490d-b825-bb32f97cbb57 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:05.522254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0cd75656-3279-490d-b825-bb32f97cbb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:05.536908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 698/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:05.576065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f257-29d9-4023-8991-ea0d3d11d9c0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:05.591818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f257-29d9-4023-8991-ea0d3d11d9c0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:05.597389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:05.598095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:05.599038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:05.599686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:05.609611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:05.611332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cac7c-9675-43d1-85e4-49ad638283a6 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:05.622254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 711/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:27.145570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8916-7523-452b-aa59-4be497fa4a48 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:27.146225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8916-7523-452b-aa59-4be497fa4a48 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:27.146225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8916-7523-452b-aa59-4be497fa4a48 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:27.147567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 699/1410] 10.222.0.58 () {54 vars in 918 bytes} [Tue May 10 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:27.153192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d611d6-b333-4218-8407-50a89f95bb4e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:27.153678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d611d6-b333-4218-8407-50a89f95bb4e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:27.153798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d611d6-b333-4218-8407-50a89f95bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:27.171380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e8d611d6-b333-4218-8407-50a89f95bb4e None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:27.172572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e8d611d6-b333-4218-8407-50a89f95bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:27.181579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 712/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Tue May 10 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:27.196871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a84f-74a3-4850-89af-e9b7d36e0cad None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:27.197160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a84f-74a3-4850-89af-e9b7d36e0cad None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:27.197439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a84f-74a3-4850-89af-e9b7d36e0cad None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:27.198835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 700/1412] 10.222.0.58 () {54 vars in 918 bytes} [Tue May 10 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:27.203545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7997a-8a31-41b5-8a02-bf7ea2598380 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:27.203846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7997a-8a31-41b5-8a02-bf7ea2598380 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:27.204128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7997a-8a31-41b5-8a02-bf7ea2598380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:27.221870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8ef7997a-8a31-41b5-8a02-bf7ea2598380 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:27.223661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8ef7997a-8a31-41b5-8a02-bf7ea2598380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:27.232567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 713/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Tue May 10 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:29.922039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47693a2-036d-4bb5-b6ff-59857f1304ca None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:29.922649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47693a2-036d-4bb5-b6ff-59857f1304ca None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:29.922649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47693a2-036d-4bb5-b6ff-59857f1304ca None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:29.923978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:09:29.929454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58bf1-1dcb-4f92-aa4f-f846bb09da3f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:29.929894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58bf1-1dcb-4f92-aa4f-f846bb09da3f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:29.930051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58bf1-1dcb-4f92-aa4f-f846bb09da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:29.948424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1fb58bf1-1dcb-4f92-aa4f-f846bb09da3f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:29.949606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1fb58bf1-1dcb-4f92-aa4f-f846bb09da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:29.958019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:29.999110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47693a2-036d-4bb5-b6ff-59857f1304ca None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:30.013758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47693a2-036d-4bb5-b6ff-59857f1304ca None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:30.019895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:30.020538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:30.020823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:30.021101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:30.029030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:30.030637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebef25-bb64-42fd-9269-9d8c68dd5035 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:30.041160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 702/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:31.044218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c47d71-f08e-4a48-a226-4eead370a97a None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:31.044772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c47d71-f08e-4a48-a226-4eead370a97a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:31.044772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c47d71-f08e-4a48-a226-4eead370a97a None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:31.046019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:31.050926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0189-fbd8-47c9-afc6-0d2e2cd0a884 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:31.051399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0189-fbd8-47c9-afc6-0d2e2cd0a884 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:31.051555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0189-fbd8-47c9-afc6-0d2e2cd0a884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:31.069393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-33aa0189-fbd8-47c9-afc6-0d2e2cd0a884 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:31.070599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-33aa0189-fbd8-47c9-afc6-0d2e2cd0a884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:31.078519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:31.238112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c47d71-f08e-4a48-a226-4eead370a97a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:31.252254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c47d71-f08e-4a48-a226-4eead370a97a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:31.258133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:31.258786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:31.259068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:31.259342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:31.267640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:31.269931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181464c-8e5d-42fb-86e1-9fd411db96ff service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:31.279923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 716/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:32.236565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d50e8-be05-4ff5-bab4-61b0b1657c0c None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:32.237109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d50e8-be05-4ff5-bab4-61b0b1657c0c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:32.237109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d50e8-be05-4ff5-bab4-61b0b1657c0c None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:32.238621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:32.243729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcc4d-49c6-4213-9034-987ebf33a968 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:32.244189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcc4d-49c6-4213-9034-987ebf33a968 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:32.244289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcc4d-49c6-4213-9034-987ebf33a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:32.261987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-cf3fcc4d-49c6-4213-9034-987ebf33a968 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:32.263546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cf3fcc4d-49c6-4213-9034-987ebf33a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:32.272167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:32.423653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d50e8-be05-4ff5-bab4-61b0b1657c0c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:32.437917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d50e8-be05-4ff5-bab4-61b0b1657c0c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:32.443928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:32.444557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:32.444841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:32.445120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:32.453066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:32.454680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3932c-1505-4fda-b98d-967d6ef3f6db service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:32.464539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 705/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:33.478329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dabb71-9007-450e-873e-af416458a01d None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:33.478918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dabb71-9007-450e-873e-af416458a01d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:33.479021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dabb71-9007-450e-873e-af416458a01d None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:33.480360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:33.485897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05921-93fb-43e8-a99f-aff1051cd5e2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:33.491597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05921-93fb-43e8-a99f-aff1051cd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:33.491597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05921-93fb-43e8-a99f-aff1051cd5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:33.510569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-abb05921-93fb-43e8-a99f-aff1051cd5e2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:33.516348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-abb05921-93fb-43e8-a99f-aff1051cd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:33.520577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:33.701080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dabb71-9007-450e-873e-af416458a01d None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:33.717944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dabb71-9007-450e-873e-af416458a01d None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:33.725452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:33.726111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:33.726426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:33.726704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:33.735689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:33.737219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c94ed-ddae-4272-be92-148e1b47a38d service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:33.749800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 719/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:34.716406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8303f-b1a9-4267-92cf-8dafad873f88 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:34.717998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8303f-b1a9-4267-92cf-8dafad873f88 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:34.718372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8303f-b1a9-4267-92cf-8dafad873f88 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:34.719850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:34.725135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedaab-9af1-4673-a50c-1761bf5f4ccc None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:34.725586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedaab-9af1-4673-a50c-1761bf5f4ccc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:34.725682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedaab-9af1-4673-a50c-1761bf5f4ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:34.743768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-53eedaab-9af1-4673-a50c-1761bf5f4ccc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:34.744893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-53eedaab-9af1-4673-a50c-1761bf5f4ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:34.752738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:34.907670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8303f-b1a9-4267-92cf-8dafad873f88 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:34.921442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8303f-b1a9-4267-92cf-8dafad873f88 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:34.927025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:34.927654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:34.927932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:34.928205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:34.935963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:34.937512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c05e9-6a62-4c80-a782-ac335baa8693 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:34.947226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 708/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:35.890839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c19a-bbe6-4c5a-a3ae-cdcf53721bcc None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:35.891379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c19a-bbe6-4c5a-a3ae-cdcf53721bcc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:35.891379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c19a-bbe6-4c5a-a3ae-cdcf53721bcc None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:35.892802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:35.898418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905b28-c180-4917-91be-3aa6e6b90220 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:35.898892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905b28-c180-4917-91be-3aa6e6b90220 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:35.899004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98905b28-c180-4917-91be-3aa6e6b90220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:35.916945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-98905b28-c180-4917-91be-3aa6e6b90220 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:35.918142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-98905b28-c180-4917-91be-3aa6e6b90220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:35.926261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:36.076880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c19a-bbe6-4c5a-a3ae-cdcf53721bcc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:36.091508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c19a-bbe6-4c5a-a3ae-cdcf53721bcc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:36.097448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:36.098147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:36.098459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:36.098731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:36.106713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:36.108285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015435a1-d37a-482d-9e19-00ed59627734 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:36.118045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 722/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:37.102508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece598-891d-4852-a33b-8c035d739f2f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:37.103046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece598-891d-4852-a33b-8c035d739f2f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:37.103046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece598-891d-4852-a33b-8c035d739f2f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:37.104396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:37.109364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf78a-8a77-492c-bcd3-12e93ef6817b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:37.109796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf78a-8a77-492c-bcd3-12e93ef6817b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:37.109913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf78a-8a77-492c-bcd3-12e93ef6817b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:37.127573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-56ccf78a-8a77-492c-bcd3-12e93ef6817b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:37.129439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-56ccf78a-8a77-492c-bcd3-12e93ef6817b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:37.137250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:37.285594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece598-891d-4852-a33b-8c035d739f2f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:37.299374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece598-891d-4852-a33b-8c035d739f2f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:37.305312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:37.305934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:37.306243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:37.306520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:37.314362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:37.315987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2f9e-6555-4ac1-8fc7-9baf152459d9 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:37.325030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 711/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:38.306100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2f52-2136-4b63-b9fb-7477741a06cb None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:38.306666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2f52-2136-4b63-b9fb-7477741a06cb None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:38.306666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2f52-2136-4b63-b9fb-7477741a06cb None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:38.307973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:38.312817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d767c7-58e4-489e-8d9c-33143e24b47a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:38.313215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d767c7-58e4-489e-8d9c-33143e24b47a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:38.313384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d767c7-58e4-489e-8d9c-33143e24b47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:38.331996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-44d767c7-58e4-489e-8d9c-33143e24b47a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:38.333232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-44d767c7-58e4-489e-8d9c-33143e24b47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:38.341001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:38.497852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f2f52-2136-4b63-b9fb-7477741a06cb None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:38.511934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f2f52-2136-4b63-b9fb-7477741a06cb None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:38.517683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:38.518346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:38.518726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:38.519155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:38.528366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:38.530908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec20e1-a22d-4b14-9de4-ed42b0cf73fa service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:38.545382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 725/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:39.492952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178eaa-0fdd-4378-b0a0-bd57c0b22c0e None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:39.493556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178eaa-0fdd-4378-b0a0-bd57c0b22c0e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:39.493556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98178eaa-0fdd-4378-b0a0-bd57c0b22c0e None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:39.494889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:39.499798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed229f6-fce9-400a-818c-e8abb6cb5b3b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:39.500175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed229f6-fce9-400a-818c-e8abb6cb5b3b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:39.500442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed229f6-fce9-400a-818c-e8abb6cb5b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:39.520494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0ed229f6-fce9-400a-818c-e8abb6cb5b3b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:39.521778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0ed229f6-fce9-400a-818c-e8abb6cb5b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:39.529827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:39.684296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98178eaa-0fdd-4378-b0a0-bd57c0b22c0e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:39.698685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98178eaa-0fdd-4378-b0a0-bd57c0b22c0e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:39.704030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:39.704665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:39.704941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:39.705214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:39.714312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:39.715877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff1a32-7294-4913-a80b-0afe41b785f9 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:39.726165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 714/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:41.886741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b930-547e-4031-a1ee-43d7c5e7d4ed None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:41.887319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b930-547e-4031-a1ee-43d7c5e7d4ed None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:41.887319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b930-547e-4031-a1ee-43d7c5e7d4ed None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:41.888988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:41.894186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6cedd-33b0-421a-9da0-5c3bdf5458cc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:41.894604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6cedd-33b0-421a-9da0-5c3bdf5458cc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:41.894772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6cedd-33b0-421a-9da0-5c3bdf5458cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:41.911836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-90a6cedd-33b0-421a-9da0-5c3bdf5458cc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:41.913024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-90a6cedd-33b0-421a-9da0-5c3bdf5458cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:41.920743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:09:41.961891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b930-547e-4031-a1ee-43d7c5e7d4ed None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:41.972446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b930-547e-4031-a1ee-43d7c5e7d4ed None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:41.976019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:41.976656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:41.976931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:41.977201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:41.987858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:41.989309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff0601-2577-483e-8574-57ef39135f49 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:41.999019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 728/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:44.461578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad737bb-335b-4d37-96be-b06e80b0ba1a None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:44.462120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad737bb-335b-4d37-96be-b06e80b0ba1a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:44.462120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad737bb-335b-4d37-96be-b06e80b0ba1a None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:44.463547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:44.468883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345d36-dccc-4948-9e09-faae45c11353 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:44.473405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345d36-dccc-4948-9e09-faae45c11353 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:44.473405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345d36-dccc-4948-9e09-faae45c11353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:44.489007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-95345d36-dccc-4948-9e09-faae45c11353 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:44.490219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-95345d36-dccc-4948-9e09-faae45c11353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:44.497995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:44.537594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad737bb-335b-4d37-96be-b06e80b0ba1a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:44.548249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad737bb-335b-4d37-96be-b06e80b0ba1a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:44.552867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:44.553495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:44.553797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:44.554070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:44.562725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:44.564132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7e2bc-036c-4c9e-be3b-b899608a5215 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:44.573732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 717/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:47.267912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19b42-349d-43c6-aa61-f1b64bf5b796 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:47.268569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19b42-349d-43c6-aa61-f1b64bf5b796 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:47.268569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19b42-349d-43c6-aa61-f1b64bf5b796 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:47.270410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:47.275713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf498-b288-49b8-a35b-ac95f4582859 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:47.276238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf498-b288-49b8-a35b-ac95f4582859 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:47.276349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf498-b288-49b8-a35b-ac95f4582859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:47.294504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5eeaf498-b288-49b8-a35b-ac95f4582859 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:47.295673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5eeaf498-b288-49b8-a35b-ac95f4582859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:47.303847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:47.341553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec19b42-349d-43c6-aa61-f1b64bf5b796 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:47.353147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec19b42-349d-43c6-aa61-f1b64bf5b796 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:47.356644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:47.357271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:47.357575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:47.357843 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:47.366740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:47.368154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3132d3e-6120-4a45-8b05-6fea23420eb6 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:47.377204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 731/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:49.756211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c70ad-0492-4648-864f-93ea5956e2da None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:49.756754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c70ad-0492-4648-864f-93ea5956e2da None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:49.756754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c70ad-0492-4648-864f-93ea5956e2da None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:49.758225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:49.763697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf649f-eef2-4bdf-aade-9c0faacd737c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:49.764107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf649f-eef2-4bdf-aade-9c0faacd737c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:49.764247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf649f-eef2-4bdf-aade-9c0faacd737c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:49.782320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8fcf649f-eef2-4bdf-aade-9c0faacd737c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:49.783549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8fcf649f-eef2-4bdf-aade-9c0faacd737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:49.791873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:49.829662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c70ad-0492-4648-864f-93ea5956e2da None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:49.840351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c70ad-0492-4648-864f-93ea5956e2da None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:49.843922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:49.844554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:49.844847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:49.845149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:49.854424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:49.855854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ed351-9c18-48c2-9d94-409deb0f5c32 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:49.865510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 720/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:52.187604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e9c4-199f-4347-a294-b5e5396d6440 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:52.188192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e9c4-199f-4347-a294-b5e5396d6440 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:52.188192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e9c4-199f-4347-a294-b5e5396d6440 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:52.189522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:52.195748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72e7a2-3ce8-4d20-9fc0-a9cb0ae97c61 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:52.196212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72e7a2-3ce8-4d20-9fc0-a9cb0ae97c61 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:52.196319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72e7a2-3ce8-4d20-9fc0-a9cb0ae97c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:52.214919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2c72e7a2-3ce8-4d20-9fc0-a9cb0ae97c61 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:52.216108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2c72e7a2-3ce8-4d20-9fc0-a9cb0ae97c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:52.224246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:52.261948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28e9c4-199f-4347-a294-b5e5396d6440 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:52.273232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28e9c4-199f-4347-a294-b5e5396d6440 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:52.276681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:52.277306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:52.277589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:52.277860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:52.286829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:52.288311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83217616-e321-4828-bd7a-831cff6946f7 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:52.298016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 734/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:54.587431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d680e6-f2f2-4999-991a-b9d77b18bc88 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:54.588014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d680e6-f2f2-4999-991a-b9d77b18bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:54.588014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d680e6-f2f2-4999-991a-b9d77b18bc88 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:54.589347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:54.594699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3820cc-ee6a-4523-90c2-78ab7453be09 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:54.595121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3820cc-ee6a-4523-90c2-78ab7453be09 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:54.595264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3820cc-ee6a-4523-90c2-78ab7453be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:54.613509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8a3820cc-ee6a-4523-90c2-78ab7453be09 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:54.614737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8a3820cc-ee6a-4523-90c2-78ab7453be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:54.622589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:54.659962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d680e6-f2f2-4999-991a-b9d77b18bc88 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:54.670417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d680e6-f2f2-4999-991a-b9d77b18bc88 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:54.673834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:54.675080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:54.675455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:54.675731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:54.684284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:54.685696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7247f-57df-479f-bee7-9c64351f35ea service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:54.695822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 723/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:56.950612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:56.951207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:56.951207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:56.952534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:56.958563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f497ec-406c-4fba-a030-06e4e4aba009 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:56.959002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f497ec-406c-4fba-a030-06e4e4aba009 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:56.959141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f497ec-406c-4fba-a030-06e4e4aba009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:56.977187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f1f497ec-406c-4fba-a030-06e4e4aba009 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:56.978365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f1f497ec-406c-4fba-a030-06e4e4aba009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:56.986126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:57.023766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:57.025473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:57.042213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68214dc3-f14d-467e-8b67-1299a214066a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:57.045702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:57.046369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:57.046651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:57.046922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:57.056216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:57.057644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbe44c-c315-42d4-8d1e-4e4503ef57f5 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:57.068124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 737/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:09:59.347295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c357b-1178-4cad-9b8f-547b29324409 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:59.347878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c357b-1178-4cad-9b8f-547b29324409 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:59.347878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c357b-1178-4cad-9b8f-547b29324409 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:59.349249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:09:59.354553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7eb4e-56c2-4322-adf6-f224b5d96549 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:59.354984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7eb4e-56c2-4322-adf6-f224b5d96549 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:59.355110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7eb4e-56c2-4322-adf6-f224b5d96549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:59.372685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d5a7eb4e-56c2-4322-adf6-f224b5d96549 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:09:59.373851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d5a7eb4e-56c2-4322-adf6-f224b5d96549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:09:59.382559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:09:59.420550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c357b-1178-4cad-9b8f-547b29324409 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:09:59.432476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c357b-1178-4cad-9b8f-547b29324409 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:09:59.435939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:09:59.436564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:09:59.436842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:09:59.437115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:09:59.446711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:09:59.448121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dee00-85c5-4ad5-8c16-9f02ad22bda3 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:09:59.457447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 726/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:10:01.717300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf6119-f832-41d5-a8c1-40aceabeb259 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:01.717860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf6119-f832-41d5-a8c1-40aceabeb259 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:01.717860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf6119-f832-41d5-a8c1-40aceabeb259 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:01.719272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:01.724889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418eee1-4713-4189-9720-17ff63df3067 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:01.725351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418eee1-4713-4189-9720-17ff63df3067 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:01.725462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418eee1-4713-4189-9720-17ff63df3067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:01.742849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0418eee1-4713-4189-9720-17ff63df3067 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:01.744019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0418eee1-4713-4189-9720-17ff63df3067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:01.751740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:01.788170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf6119-f832-41d5-a8c1-40aceabeb259 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:01.798692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf6119-f832-41d5-a8c1-40aceabeb259 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:01.802356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:01.802991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:01.803275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:01.803551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:01.812191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:01.813616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044a63-9e9d-49d1-af56-643964e71c11 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:01.823486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 740/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:04.034784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3008-25cd-46aa-9e2b-d2cab3e501c5 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:04.035328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3008-25cd-46aa-9e2b-d2cab3e501c5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:04.035328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee3008-25cd-46aa-9e2b-d2cab3e501c5 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:04.036649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:04.042059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d273ef-251f-46f5-81ac-91a0a39a0b62 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:04.042496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d273ef-251f-46f5-81ac-91a0a39a0b62 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:04.042659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d273ef-251f-46f5-81ac-91a0a39a0b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:04.060527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-48d273ef-251f-46f5-81ac-91a0a39a0b62 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:04.061711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-48d273ef-251f-46f5-81ac-91a0a39a0b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:04.069807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:04.105771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee3008-25cd-46aa-9e2b-d2cab3e501c5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:04.117059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee3008-25cd-46aa-9e2b-d2cab3e501c5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:04.120703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:04.121330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:04.121609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:04.121881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:04.130859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:04.132276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a17dc-1607-4435-8544-15f502041fdd service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:04.141611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 729/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:38.738541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303db5-e4c9-447a-8df6-a885ce98cb41 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:38.739073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303db5-e4c9-447a-8df6-a885ce98cb41 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:38.739199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303db5-e4c9-447a-8df6-a885ce98cb41 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:38.740602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 742/1471] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:38.745579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b3c7-583a-465e-9eae-a1df9a751188 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:38.746048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b3c7-583a-465e-9eae-a1df9a751188 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:38.746154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2b3c7-583a-465e-9eae-a1df9a751188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:38.764257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fcf2b3c7-583a-465e-9eae-a1df9a751188 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:38.765435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fcf2b3c7-583a-465e-9eae-a1df9a751188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:38.773730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 730/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:38.821490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52132b31-711f-4c91-b3d8-87ac09e8292b None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:38.821772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52132b31-711f-4c91-b3d8-87ac09e8292b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:38.822045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52132b31-711f-4c91-b3d8-87ac09e8292b None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:38.823456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 743/1473] 10.222.0.58 () {54 vars in 962 bytes} [Tue May 10 14:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:38.828162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508856a-7815-42ed-9298-a5e7db78f522 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:38.828448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508856a-7815-42ed-9298-a5e7db78f522 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:38.828721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508856a-7815-42ed-9298-a5e7db78f522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:38.846699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6508856a-7815-42ed-9298-a5e7db78f522 None None] MFA Rules not processed for user `4ee88945608d42c0981bb93470e832fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:38.847861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6508856a-7815-42ed-9298-a5e7db78f522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:38.854126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 731/1474] 10.222.0.58 () {58 vars in 1059 bytes} [Tue May 10 14:10:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:38.861679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52132b31-711f-4c91-b3d8-87ac09e8292b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:38.871950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52132b31-711f-4c91-b3d8-87ac09e8292b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:38.875364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:38.875984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:38.876923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:38.877196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:38.885715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:38.887160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084279f-03d6-4dc7-868c-129bb6e395c9 service glance] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:38.896588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 744/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:10:41.218351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393187-cfbe-475c-b79c-a3642edc4f45 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:41.218912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393187-cfbe-475c-b79c-a3642edc4f45 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:41.218912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e393187-cfbe-475c-b79c-a3642edc4f45 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:41.221040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 732/1476] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:41.226460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec12b1-f389-47b7-9bd6-bd989b293186 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:41.226857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec12b1-f389-47b7-9bd6-bd989b293186 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:41.227024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec12b1-f389-47b7-9bd6-bd989b293186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:41.244351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f0ec12b1-f389-47b7-9bd6-bd989b293186 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:41.245561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f0ec12b1-f389-47b7-9bd6-bd989b293186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:41.253713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 745/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:41.277796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393187-cfbe-475c-b79c-a3642edc4f45 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:41.291133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e393187-cfbe-475c-b79c-a3642edc4f45 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:41.295598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:41.296241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:41.296525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:41.296805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:41.297534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:41.299357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:10:41.299357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:10:41.299357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:10:41.300507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9655dc-41ed-4e88-916f-4896aa7a8954 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:41.319194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 733/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Tue May 10 14:10:41 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:10:43.562520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c913-32e1-44d7-895b-a08ec3255d69 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:43.563085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c913-32e1-44d7-895b-a08ec3255d69 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:43.563085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c913-32e1-44d7-895b-a08ec3255d69 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:43.565513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 746/1479] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:43.571113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6527d90-5c93-4db9-839b-92fc04905a02 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:43.571529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6527d90-5c93-4db9-839b-92fc04905a02 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:43.571731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6527d90-5c93-4db9-839b-92fc04905a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:43.589908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e6527d90-5c93-4db9-839b-92fc04905a02 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:43.591183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e6527d90-5c93-4db9-839b-92fc04905a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:43.599055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 734/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:43.642047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5c913-32e1-44d7-895b-a08ec3255d69 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:43.655948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5c913-32e1-44d7-895b-a08ec3255d69 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:43.661719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:43.662374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:43.662657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:43.662931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:43.670723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:43.672292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423de46c-1c1c-4fb9-929c-78d33bce8220 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:43.682093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 747/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:46.084128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8dc6-6791-43de-8ea2-376b5b7c63e9 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:46.084656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8dc6-6791-43de-8ea2-376b5b7c63e9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:46.084656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e8dc6-6791-43de-8ea2-376b5b7c63e9 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:46.086039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 735/1482] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:46.091404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4f70a-9f62-4c9c-a5ec-a976df37e1b6 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:46.091824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4f70a-9f62-4c9c-a5ec-a976df37e1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:46.091946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4f70a-9f62-4c9c-a5ec-a976df37e1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:46.109480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ace4f70a-9f62-4c9c-a5ec-a976df37e1b6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:46.110846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ace4f70a-9f62-4c9c-a5ec-a976df37e1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:46.118530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 748/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:46.154391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e8dc6-6791-43de-8ea2-376b5b7c63e9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:46.164927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e8dc6-6791-43de-8ea2-376b5b7c63e9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:46.168379 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:46.169002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:46.169281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:46.169553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:46.178303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:46.179719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ddb01-03ab-4a68-a60d-750b82700bd4 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:46.189498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 736/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:48.477623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff3719-355d-4a88-a0e9-2d3c99abb739 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:48.478276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff3719-355d-4a88-a0e9-2d3c99abb739 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:48.478276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff3719-355d-4a88-a0e9-2d3c99abb739 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:48.479575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 749/1485] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:48.484767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a020bb-8772-4726-80fa-cdd4da692f16 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:48.485801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a020bb-8772-4726-80fa-cdd4da692f16 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:48.485947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a020bb-8772-4726-80fa-cdd4da692f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:48.503484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-57a020bb-8772-4726-80fa-cdd4da692f16 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:48.516743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-57a020bb-8772-4726-80fa-cdd4da692f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:48.517994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 737/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:48.557326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff3719-355d-4a88-a0e9-2d3c99abb739 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:48.568023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff3719-355d-4a88-a0e9-2d3c99abb739 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:48.571567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:48.572191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:48.572470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:48.572742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:48.582543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:48.584003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc579-b0b7-4322-bf1b-9f34f1322c43 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:48.593326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 750/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:51.035240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c660-0f56-4680-8295-fa4f50c48be4 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:51.035790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c660-0f56-4680-8295-fa4f50c48be4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:51.035790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c660-0f56-4680-8295-fa4f50c48be4 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:51.037088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 738/1488] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:51.042418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff58c-e916-4d6c-bf99-6d6124d30539 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:51.042823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff58c-e916-4d6c-bf99-6d6124d30539 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:51.042966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff58c-e916-4d6c-bf99-6d6124d30539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:51.061274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-394ff58c-e916-4d6c-bf99-6d6124d30539 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:51.062484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-394ff58c-e916-4d6c-bf99-6d6124d30539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:51.071204 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 751/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:51.111169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c660-0f56-4680-8295-fa4f50c48be4 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:51.121639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c660-0f56-4680-8295-fa4f50c48be4 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:51.125121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:51.126506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:51.126788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:51.127066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:51.135713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:51.137130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056571b-f8d8-4309-8ec3-8e2fe770b599 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:51.146748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 739/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:10:53.873760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681073c-7771-4bb5-99ff-2b7c7d3bd33f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:53.874329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681073c-7771-4bb5-99ff-2b7c7d3bd33f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:53.874329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681073c-7771-4bb5-99ff-2b7c7d3bd33f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:53.875751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 752/1491] 10.222.0.58 () {54 vars in 926 bytes} [Tue May 10 14:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:10:53.881751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5719-cb89-40ad-91dd-4490b07f7d47 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:53.882158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5719-cb89-40ad-91dd-4490b07f7d47 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:53.882349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd5719-cb89-40ad-91dd-4490b07f7d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:53.899741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-55bd5719-cb89-40ad-91dd-4490b07f7d47 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:10:53.900921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-55bd5719-cb89-40ad-91dd-4490b07f7d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:10:53.908996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 740/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Tue May 10 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:10:53.947270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681073c-7771-4bb5-99ff-2b7c7d3bd33f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:10:53.958404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681073c-7771-4bb5-99ff-2b7c7d3bd33f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:10:53.961939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:10:53.962603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:10:53.962885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:10:53.963157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:10:53.971950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:10:53.973341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9cbca-36dd-40af-bb4e-459c4f612806 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:10:53.982478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 753/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:12:32.929728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:32.930506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:32.930506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:32.947550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:12:32.948722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:32.956315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 741/1494] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:12:32.965592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:32.966126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:32.966126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:32.985610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:12:32.986772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:32.995185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 754/1495] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:12:32.999816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.009275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908e96-9a35-4b25-bf1e-b49900ce76b9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.013730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.014402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.014690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.014969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.015698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:12:33.016005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.017267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.017267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:12:33.017267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:12:33.018175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.030117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 742/1496] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:12:33.038576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.048421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7141bda-5ef2-406a-baa9-7d0a4db6d9b7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.053042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.053689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.054119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.054906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.055614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.057467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.057467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:12:33.057467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:12:33.058701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.164390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 755/1497] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:12:33 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:12:33.171929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.182231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e9b-2e9b-434c-9ff2-9a7ff9ce1af2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.237965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.238586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.238867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.239139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.239827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.240817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.240817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:12:33.240817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:12:33.242370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.333697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 743/1498] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:12:33 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:12:33.338567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.348872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d5a3-cd9f-4adc-a5cc-db4a9b903cfa admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.354536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.355196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.355479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.355755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.356479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.357489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.357489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:12:33.357489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:12:33.358394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.371254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 756/1499] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:12:33.375196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.385579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d6fc5-f453-4810-bdc2-b669eb85770a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.390163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.390842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.391129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.391665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.392392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.393391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.393391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:12:33.393391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:12:33.394319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c11a6-b986-409a-af18-b38717e1a75f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.410136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 744/1500] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:12:33.414373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.424305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb7573-1043-4dbb-b550-0302356e20bc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.428733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.429859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.430140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.430443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] PATH_INFO: `/v3/projects/f0f65b685da94b4e8847ee70051b1c32/users/1dba9c637b5c437cb261b120d3e43923/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.445405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f65b685da94b4e8847ee70051b1c32, user_id=1dba9c637b5c437cb261b120d3e43923, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.446461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.446461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:12:33.446461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:12:33.448422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef14a4-9c73-4197-bf1c-820f6800c6c7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.537806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 757/1501] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:12:33 2022] PUT /identity/v3/projects/f0f65b685da94b4e8847ee70051b1c32/users/1dba9c637b5c437cb261b120d3e43923/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:12:33.549485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d1465-4551-4ea5-8a05-f79a9b86dc0d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.552068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d1465-4551-4ea5-8a05-f79a9b86dc0d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.552068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d1465-4551-4ea5-8a05-f79a9b86dc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.577855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-719d1465-4551-4ea5-8a05-f79a9b86dc0d None None] MFA Rules not processed for user `1dba9c637b5c437cb261b120d3e43923`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:12:33.579102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-719d1465-4551-4ea5-8a05-f79a9b86dc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:33.647020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 745/1502] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:12:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:12:33.660758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6820-c469-44a4-ad67-4d547c26b518 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.661058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6820-c469-44a4-ad67-4d547c26b518 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.661339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6820-c469-44a4-ad67-4d547c26b518 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.662792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 758/1503] 10.222.0.58 () {54 vars in 963 bytes} [Tue May 10 14:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:12:33.668505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7e55-2912-434b-bc27-888403a764ab None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.668787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7e55-2912-434b-bc27-888403a764ab None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.669119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7e55-2912-434b-bc27-888403a764ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.687548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-aa1f7e55-2912-434b-bc27-888403a764ab None None] MFA Rules not processed for user `0ef044e8d1fb4953b2cc3244f4a4369f`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:12:33.688739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-aa1f7e55-2912-434b-bc27-888403a764ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:33.741376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 746/1504] 10.222.0.58 () {58 vars in 1060 bytes} [Tue May 10 14:12:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:12:33.749840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6820-c469-44a4-ad67-4d547c26b518 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.760837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6820-c469-44a4-ad67-4d547c26b518 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.764317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.764945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.765223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.765492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.774873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.776291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:33.784827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 759/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:12:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:12:33.827271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d9587-38bc-4916-9ea2-46504879cfe3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.827565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d9587-38bc-4916-9ea2-46504879cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.827876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d9587-38bc-4916-9ea2-46504879cfe3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.829176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 747/1506] 10.222.0.58 () {52 vars in 829 bytes} [Tue May 10 14:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:12:33.832836 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:33.893115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea8cec-c966-4714-a04f-f381b8c5e973 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:33.898106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:33.898989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:33.899270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:33.899546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] PATH_INFO: `/v3/users/1dba9c637b5c437cb261b120d3e43923` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:33.900727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dba9c637b5c437cb261b120d3e43923)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:33.901743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:33.901743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:12:33.901743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:12:33.903070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:34.181162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dba9c637b5c437cb261b120d3e43923'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:34.181934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dba9c637b5c437cb261b120d3e43923'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:34.200090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dba9c637b5c437cb261b120d3e43923'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:34.212624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dba9c637b5c437cb261b120d3e43923'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:34.213404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dba9c637b5c437cb261b120d3e43923'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:34.310714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 760/1507] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:12:33 2022] DELETE /identity/v3/users/1dba9c637b5c437cb261b120d3e43923 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:12:34.315261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:34.315722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:34.315853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:34.333416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:12:34.334643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:34.400540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 748/1508] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:12:34.410054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:34.411496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:34.466665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2c783-e70f-4d9e-9056-ad28f13ce35e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:34.470124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:34.470797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:34.471087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:34.471361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:34.480220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:34.481660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd28f7e-d834-4b7a-b72d-d7f311e964ea service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:34.490900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 761/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:12:34.962786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:12:34.964218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:12:34.976045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0313396-4cd2-4d3f-949a-16b2079bd730 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:12:34.981171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:12:34.982338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:12:34.982626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:12:34.982906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] PATH_INFO: `/v3/projects/f0f65b685da94b4e8847ee70051b1c32` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:12:34.984375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f65b685da94b4e8847ee70051b1c32)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:12:34.985712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:12:34.985712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:12:34.985712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:12:34.986716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:12:35.138941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f65b685da94b4e8847ee70051b1c32', 'request_id': 'req-364b46eb-c830-454f-85d3-4af9c044ae1a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:35.402752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f65b685da94b4e8847ee70051b1c32', 'request_id': 'req-364b46eb-c830-454f-85d3-4af9c044ae1a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:35.409271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f65b685da94b4e8847ee70051b1c32', 'request_id': 'req-364b46eb-c830-454f-85d3-4af9c044ae1a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:35.429012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] The token cache is being invalidate because project f0f65b685da94b4e8847ee70051b1c32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:12:35.429300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:35.429956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-364b46eb-c830-454f-85d3-4af9c044ae1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:12:35.432086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 749/1510] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:12:34 2022] DELETE /identity/v3/projects/f0f65b685da94b4e8847ee70051b1c32 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:14:20.743927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484ad78-6505-4e97-b729-61071916b31c None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:20.744479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484ad78-6505-4e97-b729-61071916b31c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:20.744479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484ad78-6505-4e97-b729-61071916b31c None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:20.745842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:20.750667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd178ae-bb7e-482c-ac7c-1b755647896f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:20.751125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd178ae-bb7e-482c-ac7c-1b755647896f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:20.751224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd178ae-bb7e-482c-ac7c-1b755647896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:20.768832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0dd178ae-bb7e-482c-ac7c-1b755647896f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:20.770019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0dd178ae-bb7e-482c-ac7c-1b755647896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:20.828405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:20.999532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484ad78-6505-4e97-b729-61071916b31c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:21.001135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9484ad78-6505-4e97-b729-61071916b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:21.073245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484ad78-6505-4e97-b729-61071916b31c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:21.079466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:21.080109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:21.080391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:21.080666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:21.088725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:21.091054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc83d-c1aa-4f3d-a750-3022c423aa3a service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:21.100583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 763/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:21.888312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076462c-f011-4ec3-9f5a-71e464057437 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:21.888877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076462c-f011-4ec3-9f5a-71e464057437 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:21.888877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076462c-f011-4ec3-9f5a-71e464057437 None None] PATH_INFO: `/v3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:21.890255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 751/1514] 10.222.0.64 () {54 vars in 925 bytes} [Tue May 10 14:14:21 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:14:21.897146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d737f2-725a-4b21-8f73-bb847f612c4b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:21.897602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d737f2-725a-4b21-8f73-bb847f612c4b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:21.897709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d737f2-725a-4b21-8f73-bb847f612c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:21.918430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f7d737f2-725a-4b21-8f73-bb847f612c4b None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:21.919652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f7d737f2-725a-4b21-8f73-bb847f612c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:21.985453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 764/1515] 10.222.0.64 () {58 vars in 1015 bytes} [Tue May 10 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:22.021183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03012116-2610-456b-a628-fc189d41edac None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:22.021647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03012116-2610-456b-a628-fc189d41edac None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:22.022060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03012116-2610-456b-a628-fc189d41edac None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:22.024440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 752/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:14:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:22.029490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7c47-cb09-44f9-87a9-e70b7da61f87 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:22.029782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7c47-cb09-44f9-87a9-e70b7da61f87 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:22.030089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7c47-cb09-44f9-87a9-e70b7da61f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:22.047943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-19fb7c47-cb09-44f9-87a9-e70b7da61f87 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:22.049142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-19fb7c47-cb09-44f9-87a9-e70b7da61f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:22.056967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 765/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:14:22.208867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03012116-2610-456b-a628-fc189d41edac None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:22.223218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03012116-2610-456b-a628-fc189d41edac None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:22.228659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:22.229309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:22.229594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:22.229872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:22.238809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:22.240344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae0e52-3343-4aa1-bbdc-7196d8df6007 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:22.250346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 753/1518] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:31.357282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc6884-b38c-4328-93e0-ab65e3176e9f None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:31.357839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc6884-b38c-4328-93e0-ab65e3176e9f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:31.357839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc6884-b38c-4328-93e0-ab65e3176e9f None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:31.359169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 766/1519] 10.222.0.58 () {54 vars in 960 bytes} [Tue May 10 14:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:31.363986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2622d-a302-40a7-8d92-7fe7bee87a43 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:31.364402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2622d-a302-40a7-8d92-7fe7bee87a43 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:31.364581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2622d-a302-40a7-8d92-7fe7bee87a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:31.382209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5ec2622d-a302-40a7-8d92-7fe7bee87a43 None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:31.383435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5ec2622d-a302-40a7-8d92-7fe7bee87a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:31.392458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 754/1520] 10.222.0.58 () {58 vars in 1057 bytes} [Tue May 10 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:31.399996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc6884-b38c-4328-93e0-ab65e3176e9f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:31.413890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc6884-b38c-4328-93e0-ab65e3176e9f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:31.418777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:31.419396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:31.419676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:31.419949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:31.428375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:31.429940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfe7c-5e82-48ee-a89b-08ef60685998 service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:31.439481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 767/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:33.573960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef253da-c822-422e-a089-2c1915b78bbe None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:33.574564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef253da-c822-422e-a089-2c1915b78bbe None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:33.574564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef253da-c822-422e-a089-2c1915b78bbe None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:33.575944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:33.581303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc0974-a429-4250-9003-1edf4bd5bf59 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:33.581695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc0974-a429-4250-9003-1edf4bd5bf59 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:33.581863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc0974-a429-4250-9003-1edf4bd5bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:33.599796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d2fc0974-a429-4250-9003-1edf4bd5bf59 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:33.600970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d2fc0974-a429-4250-9003-1edf4bd5bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:33.609494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:33.760770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef253da-c822-422e-a089-2c1915b78bbe None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:33.775145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef253da-c822-422e-a089-2c1915b78bbe None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:33.780353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:33.781126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:33.781413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:33.781687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:33.789491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:33.791134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9caad-1d6d-4dc3-808c-9b7887622f14 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:33.800954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 756/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:34.706261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51374013-80fa-4fe6-9810-133bab7c6607 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:34.706816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51374013-80fa-4fe6-9810-133bab7c6607 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:34.706816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51374013-80fa-4fe6-9810-133bab7c6607 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:34.708352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 10 14:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:34.713311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fc446-ecc2-4a6c-9c4b-fbebc1db923a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:34.713708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fc446-ecc2-4a6c-9c4b-fbebc1db923a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:34.713863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fc446-ecc2-4a6c-9c4b-fbebc1db923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:34.733940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-788fc446-ecc2-4a6c-9c4b-fbebc1db923a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:34.735385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-788fc446-ecc2-4a6c-9c4b-fbebc1db923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:34.743154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 10 14:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:34.897412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51374013-80fa-4fe6-9810-133bab7c6607 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:34.911572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51374013-80fa-4fe6-9810-133bab7c6607 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:34.916816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba404-0962-44f3-843a-af024372a033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:34.917441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba404-0962-44f3-843a-af024372a033 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:34.917726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba404-0962-44f3-843a-af024372a033 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:34.917996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba404-0962-44f3-843a-af024372a033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:34.925821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba404-0962-44f3-843a-af024372a033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:34.927496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba404-0962-44f3-843a-af024372a033 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:34.936569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 770/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:39.089099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:39.089639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:39.089639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:39.091114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:39.096942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416331b8-5667-4fed-b527-8a430f5cb71f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:39.097385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416331b8-5667-4fed-b527-8a430f5cb71f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:39.097541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416331b8-5667-4fed-b527-8a430f5cb71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:39.114967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-416331b8-5667-4fed-b527-8a430f5cb71f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:39.116153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-416331b8-5667-4fed-b527-8a430f5cb71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:39.123990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:39.165158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:39.166913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:39.222568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b5ac8-9b29-4452-9864-1732cdb8f48f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:39.226114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:39.226786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:39.227071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:39.227348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:39.236217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:39.237625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecf4de-d2fe-48eb-b360-55cf35b2c0d0 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:39.246777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 759/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:41.541144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:41.541697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:41.541830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:41.543295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:14:41.549106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace30a-1489-41bf-b0b2-7111e3d36b60 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:41.549586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace30a-1489-41bf-b0b2-7111e3d36b60 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:41.549687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace30a-1489-41bf-b0b2-7111e3d36b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:41.568393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4bace30a-1489-41bf-b0b2-7111e3d36b60 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:41.569591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4bace30a-1489-41bf-b0b2-7111e3d36b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:41.577372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:41.617242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:41.619280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:41.636820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e5183-10fd-4e71-a47d-5c94e04767bb None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:41.641262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:41.643588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:41.643891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:41.644171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:41.660977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:41.663047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cb124-065d-4994-a071-5d38394c0988 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:41.675407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 773/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:45.661702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:45.662301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:45.662301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:45.663689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:45.668867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b081-76ff-4249-9509-cd702a8dd6d5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:45.670866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b081-76ff-4249-9509-cd702a8dd6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:45.670866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b081-76ff-4249-9509-cd702a8dd6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:45.687635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3b13b081-76ff-4249-9509-cd702a8dd6d5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:45.688832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3b13b081-76ff-4249-9509-cd702a8dd6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:45.697058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:45.973118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:45.975198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:46.030850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9785d0-e06f-4173-971a-5ad949a5b59b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:46.034509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:46.035144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:46.035425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:46.035699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:46.044451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:46.045890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f70e2-8981-4fa9-95b2-1513e0a15f76 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:46.055049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 762/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:14:48.349991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:48.350588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:48.350588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:48.351929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 10 14:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:14:48.357106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572d109-7a9c-41c0-959f-37e79b36746a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:48.357555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572d109-7a9c-41c0-959f-37e79b36746a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:48.357653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572d109-7a9c-41c0-959f-37e79b36746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:48.376173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0572d109-7a9c-41c0-959f-37e79b36746a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:14:48.377563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0572d109-7a9c-41c0-959f-37e79b36746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:48.385338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 10 14:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:14:48.668673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:14:48.670395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:14:48.683343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579b97-ad34-4da1-90a9-d4dc9fec7e64 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:14:48.686752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:14:48.687377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:14:48.687655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:14:48.687928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:14:48.696784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:14:48.698193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7293a2-4c9f-43d7-84b2-ea8f866dbd8b service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:14:48.708087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 776/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:22.883979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2de5f-2b27-4586-8968-10aaf5667f5f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:22.884618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2de5f-2b27-4586-8968-10aaf5667f5f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:22.884618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2de5f-2b27-4586-8968-10aaf5667f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:22.930486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cfa2de5f-2b27-4586-8968-10aaf5667f5f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:22.931825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cfa2de5f-2b27-4586-8968-10aaf5667f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:22.934070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dcb5-1cfd-4af9-9fc6-5f62b49fdb85 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:22.934070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dcb5-1cfd-4af9-9fc6-5f62b49fdb85 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:22.934070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dcb5-1cfd-4af9-9fc6-5f62b49fdb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:22.950384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 764/1540] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:22.968183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-19d3dcb5-1cfd-4af9-9fc6-5f62b49fdb85 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:22.969536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-19d3dcb5-1cfd-4af9-9fc6-5f62b49fdb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:22.976061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:22.976061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:22.976356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:22.991215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 777/1541] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:22.992806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324df9-2a99-47b4-bfbe-cee14d3954a5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:22.992806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324df9-2a99-47b4-bfbe-cee14d3954a5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:22.993120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76324df9-2a99-47b4-bfbe-cee14d3954a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.011276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.012516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.022018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-76324df9-2a99-47b4-bfbe-cee14d3954a5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.023369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-76324df9-2a99-47b4-bfbe-cee14d3954a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.039886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 778/1542] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.041669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2624aa-660b-45dc-9e27-54f9f2cfeb12 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.041962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2624aa-660b-45dc-9e27-54f9f2cfeb12 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.042266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2624aa-660b-45dc-9e27-54f9f2cfeb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.042385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 765/1543] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.050004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.071595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e936448-21d9-4fce-bdeb-30ffd93e59e8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.072565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1e2624aa-660b-45dc-9e27-54f9f2cfeb12 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.073825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1e2624aa-660b-45dc-9e27-54f9f2cfeb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.089300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.089747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 779/1544] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.090684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.091094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e97af-78f1-43da-ab5e-d8b39a30a799 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.091386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e97af-78f1-43da-ab5e-d8b39a30a799 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.091667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e97af-78f1-43da-ab5e-d8b39a30a799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.091775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.092252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.093201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.093721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.096851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b09376-b243-421c-b09c-8fea0ec43eba admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.110873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 766/1545] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.112740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337580-0c62-4a7e-87b5-4504528fde5d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.113839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337580-0c62-4a7e-87b5-4504528fde5d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.114473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337580-0c62-4a7e-87b5-4504528fde5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.119451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-501e97af-78f1-43da-ab5e-d8b39a30a799 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.120674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-501e97af-78f1-43da-ab5e-d8b39a30a799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.133091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 780/1546] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.134498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.134736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.135021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.136282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-70337580-0c62-4a7e-87b5-4504528fde5d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.137781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-70337580-0c62-4a7e-87b5-4504528fde5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.152225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 767/1547] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.154954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.155244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.155523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.161026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.162558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.171069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 781/1548] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.171949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.175443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.176644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.184752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c5822-3c4f-49a3-89c7-a43c365bfcce None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.189148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 768/1549] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.190027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.195005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.195788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.196082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.196368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.197113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.197517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.198783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:23.198783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:16:23.198783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:16:23.199595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.200437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0cb12-f121-4eb1-9854-b73f224b14c7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.206047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.206901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.207186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.207464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.208354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.209362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:23.209362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:16:23.209362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:16:23.210582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85462118-9e48-4b1e-ae99-1b26d1ce7f57 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.212225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 782/1550] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.213294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.223881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061514cf-6257-472b-b521-05b1dac1a100 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.228473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.229127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.229410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.229687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.230443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.230754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.232492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1ab2-4115-42ae-a4a7-88880d6f934b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.244857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 783/1551] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.248642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babdb6-4e62-466a-bb08-8ca2615464a5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.248642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babdb6-4e62-466a-bb08-8ca2615464a5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.248642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babdb6-4e62-466a-bb08-8ca2615464a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.264280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-92babdb6-4e62-466a-bb08-8ca2615464a5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.265476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-92babdb6-4e62-466a-bb08-8ca2615464a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.273590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 784/1552] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.275221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044b6c9-adc1-4bf1-b969-62d63c58b09a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.275221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044b6c9-adc1-4bf1-b969-62d63c58b09a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.275450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044b6c9-adc1-4bf1-b969-62d63c58b09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.294155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f044b6c9-adc1-4bf1-b969-62d63c58b09a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.295452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f044b6c9-adc1-4bf1-b969-62d63c58b09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.307142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 785/1553] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.308526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25ecf-6c00-4e8b-a296-f1d1939ab7ad None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.308819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25ecf-6c00-4e8b-a296-f1d1939ab7ad None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.309097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f25ecf-6c00-4e8b-a296-f1d1939ab7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.327533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e8f25ecf-6c00-4e8b-a296-f1d1939ab7ad None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.328512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 769/1554] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.337130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e8f25ecf-6c00-4e8b-a296-f1d1939ab7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.337288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.337288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.337288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.349722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.351040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.388126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 786/1555] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.389434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.389730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.390011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.408406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.409616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.418488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 770/1556] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.419165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.422260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 787/1557] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.423122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.430878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6390a-d273-4ab7-8f52-5b05e088591a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.435347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568d2b0-9242-4d7d-a741-350af5e17455 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.435924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.436591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.436875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.437152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.438073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.440234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39079fa1-b940-458d-9cb2-aa200371d5b6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.445829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.446879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.447356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.447822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.448894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.452087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.535214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 771/1558] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.536517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.536813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.537096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.555240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.556429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.564613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 788/1559] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/projects => generated 389 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.564613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.617158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 772/1560] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.618426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.625494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0cd5e-39cd-45ee-a10d-3b845f008a5d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.627665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d97453-6fa6-4188-a284-4ff1b0218ee8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.630167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.630885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.631167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.631437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.632165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.632470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.632579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.633188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.633476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.633755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.634081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.634499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.636413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.646025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 789/1561] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.646670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.656405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604e7b0-133c-4cd2-9ed2-0105d592c633 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.660952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.661577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.661859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.662134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.662938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.663245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.664814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.676936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 790/1562] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.677697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.686934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba29a-88c1-46ea-ad6d-8df03f2b7824 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.691972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.692579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.692856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.693127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.693822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.694127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.695707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65f6f-06c5-4076-a9f8-f7f4eee228e9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.708349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 791/1563] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.709611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.709906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.710180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.728620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:23.729802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:23.733021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 773/1564] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.738465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.739187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 792/1565] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:23.743081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.744402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf54d0-7768-430b-a9d3-654f2ee84fc6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.749102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.749791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.750074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.750971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.751617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.753535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.753663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed8f64-1dea-44b5-9eda-f6af0d835089 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.758328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.758977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.759255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.759527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.760207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.761212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:23.761212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:16:23.761212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:16:23.762442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.858896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 774/1566] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.860528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.869965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0ea3-330c-4f2d-8bc7-4defafb8b4de admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.874565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.875198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.875560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.875947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.876672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:23.876986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.878616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.888840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 793/1567] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:23.890374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.891401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 775/1568] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:23.892294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:23.902822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571742-ad18-491e-a1e4-f4d62974d67f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.902999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b153f-f98a-4051-a2e0-23176c7277a5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:23.907581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.907799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:23.908230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.908527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:23.908627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.908810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:23.908968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.909152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:23.909667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.909776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:23.911595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:23.911706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.019795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 776/1569] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.023309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.035330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 794/1570] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:23 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.035330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.087557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d8ac8-5606-41a2-8e58-a1de489d3356 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.099193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df67-b32d-4abe-86d8-dca861719599 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.103824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.104495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.104774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.105049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.105767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.107419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.121129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 795/1571] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.121129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.128873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7bf4-5075-4e98-b4a0-f8fd83ff88f0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.133423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.134058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.134368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.134642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.135352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:24.135654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.137209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.142917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.143580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.143866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.144141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.144831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.146766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.149964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 796/1572] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:24.150954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.162930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc18632-c8d7-4162-aabb-429aa2de3ee9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.167691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.168356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.168655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.175164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.175164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.175164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.182711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 797/1573] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.183555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.194013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86f57-c72d-468d-bcca-482a07b8fd66 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.199224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.199999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.200309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.200585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.201310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.202962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.213911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 798/1574] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.214727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.225436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 777/1575] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.225768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2d302-1e50-4d87-9bd3-1a76c27fa48e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.225892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.284474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.285146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.285432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.285755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.286493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.287694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefc07f-136f-4303-9f0a-43befb023fd0 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.288414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.292440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.293107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.293396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.293676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.294396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.296312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.379218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 799/1576] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.380076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.409299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 778/1577] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.409299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.418787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa0195-6e21-4619-8802-81ced855c53d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.441483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362df2a-c666-41a1-a90f-fbced9bf4fe5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.446416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.447318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.447719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.447999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.448692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.450643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.474458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.478738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.478738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.478738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.478738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.478738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.488103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 779/1578] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.488884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.498137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412402-94c0-47e9-9fe7-9ad7f4c211bb admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.503161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.511383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.511383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.511383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.511383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.511383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.540327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 800/1579] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/users => generated 303 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.540565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.550386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee894-cd26-4e9b-8526-228d17d519b8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.555021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.555663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.555949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.556229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.556949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.558581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.569027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 801/1580] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.570619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.579949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf345-9781-44d0-ae66-19e235771a1f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.584536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.585157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.585444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.585723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.586458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.588014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.597405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 802/1581] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.598157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.609507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bf8d9-1bc4-4397-b559-ffb11894050c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.614015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.614681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.614963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.615234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.615895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.617788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.632892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 780/1582] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.633733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.697073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb216-c12e-4500-ad0c-067ea318d338 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.702145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.702859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.703342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.704613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.705165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.707165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.739645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 803/1583] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.740625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.750178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf790f-0539-4af2-b1a2-f83039874bbd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.755409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.756044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.756333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.756607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.757320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.758961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.769170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 804/1584] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.769991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.779585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87ad61-3b0a-496e-a435-c844fe2965dd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.784406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.785254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.785535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.785811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] PATH_INFO: `/v3/projects/8549d2b7b890411f878fe6fe776e65d7/users/9dae4d7a57f343c8bb7b92dbebea181d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.793145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 781/1585] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:24.793145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.800671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8549d2b7b890411f878fe6fe776e65d7, user_id=9dae4d7a57f343c8bb7b92dbebea181d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.800924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855df9-4293-432c-ae17-dd86e2692324 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.803382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.805500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.806152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.806479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.806780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.807487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.809082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.819539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 782/1586] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:24.820306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.830018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c44647-1230-43c5-b39e-3e50fc9684da admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.834516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.835366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.835649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.835929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] PATH_INFO: `/v3/projects/1873a08af77f4a998cf5019df662c2f2/users/0fc0746bf323456394119f58a2ad28fa/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.850692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1873a08af77f4a998cf5019df662c2f2, user_id=0fc0746bf323456394119f58a2ad28fa, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.851754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:24.851754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:16:24.851754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:16:24.853723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:24.863754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 805/1587] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:24 2022] PUT /identity/v3/projects/8549d2b7b890411f878fe6fe776e65d7/users/9dae4d7a57f343c8bb7b92dbebea181d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:24.864633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.920472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 783/1588] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:24 2022] PUT /identity/v3/projects/1873a08af77f4a998cf5019df662c2f2/users/0fc0746bf323456394119f58a2ad28fa/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:24.924294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:24.925610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eda3c8-e14b-4b15-9b57-c48c7b0dc1b5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.983513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.984309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.984591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.985456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] PATH_INFO: `/v3/projects/b8a7f6dc00b5487b9c323783ef5aaa2b/users/5e3edc87e7014d8598ba8c9517b81cf7/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.992789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276447d0-8054-4c6c-b25c-db87b7ca5529 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:24.997537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:24.998391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:24.998678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:24.998975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:24.999663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:24.999924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a7f6dc00b5487b9c323783ef5aaa2b, user_id=5e3edc87e7014d8598ba8c9517b81cf7, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.002271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.003527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.070289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 806/1589] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:24 2022] PUT /identity/v3/projects/b8a7f6dc00b5487b9c323783ef5aaa2b/users/5e3edc87e7014d8598ba8c9517b81cf7/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:25.071085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.093140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 784/1590] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:24 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:25.096886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.103649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef10f5-0b52-4746-b744-5794aa3a20f2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.130638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90774e1-4497-4edc-9283-310aeb57d8b4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.135278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.135947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.136239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.136509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.137237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.138872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.148991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 807/1591] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.149992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.159999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.160676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.160963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.161239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.161964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.162065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033d2e0-0e91-48af-86cd-35465af93964 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.163675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.166949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.167622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.167888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.168166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.168887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.170477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ddf4e-fe42-4281-a1c9-a58fbe4cc6a1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.173820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 785/1592] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.174659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.183959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 808/1593] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.183959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201937-82f3-46a2-92f2-d484fc0bc0b7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.183959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201937-82f3-46a2-92f2-d484fc0bc0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.183959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201937-82f3-46a2-92f2-d484fc0bc0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.184405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3dfc-38a3-4dc4-96c6-aae165ec3859 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.189527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.190181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.190499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.190774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.191490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.193104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259607-f9f0-4a06-b061-eff59039fedf admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.200383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-12201937-82f3-46a2-92f2-d484fc0bc0b7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.201598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-12201937-82f3-46a2-92f2-d484fc0bc0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.204828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 786/1594] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.206100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.206409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.206685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.213564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 809/1595] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:25.215072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.215470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.215967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.233677 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.234562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] MFA Rules not processed for user `0fc0746bf323456394119f58a2ad28fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.234868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.236074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.242676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 810/1596] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:25.243201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.252634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded060ce-3f87-402b-a574-cb43b7d90338 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.257294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.258011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.258328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.258604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.259332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.260929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.271383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 811/1597] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.272180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.281722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191c428-be4b-4e91-a2d6-020e8d993036 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.287168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.287808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.288095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.288374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.289089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.290704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.305228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 812/1598] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.305228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.305542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 787/1599] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:25.306133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.312437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddd656-73e6-48c0-b6dc-2ca916bba66d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.318953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.319899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e42e4-7c72-44b4-baa1-a5f338930f25 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.320441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.320842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.321129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] PATH_INFO: `/v3/projects/2a315adb7b214dceb03ed0aac50d2519/users/8d788b9560504c3795dbb623f10467cd/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.329806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.330528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.330829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.331100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.331828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.333668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.343679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a315adb7b214dceb03ed0aac50d2519, user_id=8d788b9560504c3795dbb623f10467cd, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.343829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 788/1600] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.344524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.346494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.353921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b8b0-852c-4673-9fdb-e69c45e619db admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.358476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.362378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.362378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.362378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] PATH_INFO: `/v3/projects/8d853dcac20846ed9e36e4c21b8bee20/users/2186765b0174480881be647fc9b5c3f4/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.373865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d853dcac20846ed9e36e4c21b8bee20, user_id=2186765b0174480881be647fc9b5c3f4, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.377116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.487454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 813/1601] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:25 2022] PUT /identity/v3/projects/2a315adb7b214dceb03ed0aac50d2519/users/8d788b9560504c3795dbb623f10467cd/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:25.488311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.540252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a48ec9-9fbd-41a8-8f69-20cb4994dff0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.543878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.544548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.544830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.545106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.549909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 789/1602] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:25 2022] PUT /identity/v3/projects/8d853dcac20846ed9e36e4c21b8bee20/users/2186765b0174480881be647fc9b5c3f4/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:25.560741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.605195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.605436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46eb3a5-0380-489c-bf06-4daba89a133f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.606697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.608850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.609491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.609815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.610192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.616532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 814/1603] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:25.617331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.627490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5556-51b7-45c0-9769-2c452e1ff510 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.674770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.676211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.685551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.685747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 790/1604] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:25.686484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.686875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.692427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.692427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.692427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.692427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f8a78-e2b4-4141-b8dd-4fd40577e7f4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.697396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b047d8c-8045-4724-b5fd-6055e5937fa7 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.704917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 815/1605] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:25.704917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5efce-da79-4844-bc7f-dfb1cd4142f1 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.704917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5efce-da79-4844-bc7f-dfb1cd4142f1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.704917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5efce-da79-4844-bc7f-dfb1cd4142f1 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.705641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 816/1606] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 10 14:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:25.706339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5efce-da79-4844-bc7f-dfb1cd4142f1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.706552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.707607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.708024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.708360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] PATH_INFO: `/v3/projects/6e0f048425c440288655a28aecc12ce5/users/f00f343268134dea89e0a7070da14508/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.723307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5efce-da79-4844-bc7f-dfb1cd4142f1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.725366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0f048425c440288655a28aecc12ce5, user_id=f00f343268134dea89e0a7070da14508, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.727832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.728370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39478601-a4f9-4d92-83a0-22d8c835324d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.728711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.728991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.729263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] PATH_INFO: `/v3/projects/fc0e8bf4c69e49d1b4a44905d8c9bc98/users/ecb4c97543e1452d8e6c8f1564addc76/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.744794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0e8bf4c69e49d1b4a44905d8c9bc98, user_id=ecb4c97543e1452d8e6c8f1564addc76, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.747461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a87f5-695c-40ae-bb13-a05af594e1a7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:25.847871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 817/1607] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:25 2022] PUT /identity/v3/projects/fc0e8bf4c69e49d1b4a44905d8c9bc98/users/ecb4c97543e1452d8e6c8f1564addc76/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:25.848860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 791/1608] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:25 2022] PUT /identity/v3/projects/6e0f048425c440288655a28aecc12ce5/users/f00f343268134dea89e0a7070da14508/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:25.850536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.850867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.851242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.853315 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21ac5-0d8e-45f8-88bb-5c9063debc2d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.853683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21ac5-0d8e-45f8-88bb-5c9063debc2d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.854029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21ac5-0d8e-45f8-88bb-5c9063debc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.870764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.871804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-38c21ac5-0d8e-45f8-88bb-5c9063debc2d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.872092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.873089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-38c21ac5-0d8e-45f8-88bb-5c9063debc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.936665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 818/1609] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:25.940704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 792/1610] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:25.940704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:25.943712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b688b-2e33-47eb-ab3b-3078e6e1b766 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.944212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b688b-2e33-47eb-ab3b-3078e6e1b766 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.944674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b688b-2e33-47eb-ab3b-3078e6e1b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.955083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ae5df-9c9b-4ede-8868-b60405c219b9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:25.960499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:25.961396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:25.961675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:25.961946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] PATH_INFO: `/v3/projects/3ca7cd9d55b744c88d56d22cb74f4075/users/3ee7d0ba919e4ef8bfc874c366d08595/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:25.966321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-769b688b-2e33-47eb-ab3b-3078e6e1b766 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:25.967919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-769b688b-2e33-47eb-ab3b-3078e6e1b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:25.981872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca7cd9d55b744c88d56d22cb74f4075, user_id=3ee7d0ba919e4ef8bfc874c366d08595, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:25.986188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.053173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 819/1611] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:26.054631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430304d-6808-4852-b627-f8cc807cd797 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.054993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430304d-6808-4852-b627-f8cc807cd797 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.055269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430304d-6808-4852-b627-f8cc807cd797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.074085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c430304d-6808-4852-b627-f8cc807cd797 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:26.075391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c430304d-6808-4852-b627-f8cc807cd797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:26.084157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 820/1612] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:26.085405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.085778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.086061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.101377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 793/1613] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:25 2022] PUT /identity/v3/projects/3ca7cd9d55b744c88d56d22cb74f4075/users/3ee7d0ba919e4ef8bfc874c366d08595/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:26.103107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.106107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:26.107562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:26.175006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 821/1614] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:26.175923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.183338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8146f-fc60-4d93-af0c-5e046e398e9b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.187123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a762b-fa19-4568-8636-1fc92bf7719a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.187685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.189946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.189946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.189946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.195447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.195447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.195447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.195447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.200876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.202497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.204493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.205935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.213961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 794/1615] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.214662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.216330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 822/1616] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.217429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.225999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf189a-5d81-4251-ba46-63021655b2fb service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.230318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.232972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.232972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.232972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.244498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.246271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2de08-af7a-4fb5-83b8-91ecd62ed92f service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.260747 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 823/1617] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.262113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.262406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.262810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.297313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] MFA Rules not processed for user `3ee7d0ba919e4ef8bfc874c366d08595`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:26.298748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:26.302878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690e474-e9de-496b-bd17-31329d08b16e service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.323024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.323426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.323705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.323983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.385646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 824/1618] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:26.387156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.398685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec94ba-2d79-45ad-94fb-a0b7fae494dc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.403282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.404145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.404430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.404713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.416545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.418044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.421195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.423030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff0b8d-8b4c-4ddf-8f65-19687261ca92 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.429226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 825/1619] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.429940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.435621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 795/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.502721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fee2a3-296a-4ff0-96c3-658a55f8ca0f service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.508272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.509308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.509775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.510286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.523276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.524972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a47b9-f9be-44fb-a140-08567af27b3c service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.540258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 826/1621] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:26.763405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64180f1-c958-4909-8e1c-da70603251c3 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.763772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64180f1-c958-4909-8e1c-da70603251c3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.764153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64180f1-c958-4909-8e1c-da70603251c3 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.766136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 796/1622] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 10 14:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:26.772515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf911e-9789-4f7e-8851-900192fda1c6 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.778213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf911e-9789-4f7e-8851-900192fda1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.778213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf911e-9789-4f7e-8851-900192fda1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.800791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-94cf911e-9789-4f7e-8851-900192fda1c6 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:26.802490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-94cf911e-9789-4f7e-8851-900192fda1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:26.815757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 827/1623] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:16:26.831058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64180f1-c958-4909-8e1c-da70603251c3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:26.843476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64180f1-c958-4909-8e1c-da70603251c3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:26.848892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:26.849864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:26.850342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:26.850791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:26.867646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:26.872045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd2423-d6c1-4801-b751-2625341fb466 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:26.900612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 797/1624] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:28.444239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b989e-a06c-47f6-8952-5f50ed2bc58d None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.444796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b989e-a06c-47f6-8952-5f50ed2bc58d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.444796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b989e-a06c-47f6-8952-5f50ed2bc58d None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.447534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 828/1625] 10.222.0.58 () {54 vars in 961 bytes} [Tue May 10 14:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:28.452979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400696b2-e360-4be7-8d13-4e06a31e0894 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.453605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400696b2-e360-4be7-8d13-4e06a31e0894 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.453605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400696b2-e360-4be7-8d13-4e06a31e0894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.477163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-400696b2-e360-4be7-8d13-4e06a31e0894 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:28.478922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-400696b2-e360-4be7-8d13-4e06a31e0894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:28.495192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 798/1626] 10.222.0.58 () {58 vars in 1058 bytes} [Tue May 10 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:28.496547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c9d6-c154-4430-9d56-da507aad06b0 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.496840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c9d6-c154-4430-9d56-da507aad06b0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.497190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c9d6-c154-4430-9d56-da507aad06b0 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.499253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 829/1627] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 10 14:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:28.533099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.534524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.536009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.573646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:28.575612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:28.595885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.596172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.596449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.609288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 799/1628] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:28.618557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:28.631037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a87a5c-3707-4513-a105-3cdb60c8aae1 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:28.635827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:28.638471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.639093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.639594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.648832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] MFA Rules not processed for user `2186765b0174480881be647fc9b5c3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:28.650039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:28.654156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:28.656256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:28.673944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 800/1629] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:16:28.756078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 830/1630] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:16:28.761023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:28.781177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abf97b-dc8d-4e7f-82cf-f10eb0f4e523 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:28.787834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:28.788391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:28.788679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:28.788960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:28.789664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:28.791681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:28.901353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 801/1631] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:28 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:28.901449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:28.920178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136f95-5677-44d2-ad68-92e38b1ae95f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:29.060283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:29.061077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:29.061436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:29.061787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:29.062606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:29.065411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:29.200580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 831/1632] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:28 2022] POST /identity/v3/users => generated 309 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:29.201606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:29.221158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353c19-b2c4-4f54-9c3e-1816e75d5a4c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:29.228293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:29.229039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:29.229323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:29.229603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:29.230441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:29.232119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:29.249351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:29.255811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 802/1633] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:16:29.261402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e96c1-b668-4921-aaba-e26747caed8c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:29.267555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:29.268174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:29.268464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:29.268748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] PATH_INFO: `/v3/projects/62b05d305fd145fa8d69ca029f9902b8/users/5b4b18c85fae4ea48563786cd11e7c94/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:29.287272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b05d305fd145fa8d69ca029f9902b8, user_id=5b4b18c85fae4ea48563786cd11e7c94, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:29.290079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:29.380181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 832/1634] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:29 2022] PUT /identity/v3/projects/62b05d305fd145fa8d69ca029f9902b8/users/5b4b18c85fae4ea48563786cd11e7c94/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:29.385460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:29.396634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd04aae-ddde-4d39-9f36-f63a8f4a1b22 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:29.481253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:29.482172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:29.482534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:29.483049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:29.484003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:29.486229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7587ee8-69c8-4836-9447-be38d524fcd2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:29.507927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 803/1635] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:16:29.512448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:29.530294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1155b-b309-4bd9-8d9f-ca4e521724ab admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:29.536932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:29.538522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:29.538970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:29.540483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] PATH_INFO: `/v3/projects/62b05d305fd145fa8d69ca029f9902b8/users/5b4b18c85fae4ea48563786cd11e7c94/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:29.545418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b05d305fd145fa8d69ca029f9902b8, user_id=5b4b18c85fae4ea48563786cd11e7c94, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:29.550133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc63a6-b49b-49ac-b5a0-64edf1332f6e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:29.582326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 833/1636] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:29 2022] PUT /identity/v3/projects/62b05d305fd145fa8d69ca029f9902b8/users/5b4b18c85fae4ea48563786cd11e7c94/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:30.615544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:30.616116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:30.616116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:30.662907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] MFA Rules not processed for user `f00f343268134dea89e0a7070da14508`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:30.664387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:30.792109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 804/1637] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:30.804761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc06b-e9e6-4dd1-9ab1-d3999428cc81 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:30.805688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc06b-e9e6-4dd1-9ab1-d3999428cc81 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:30.806125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc06b-e9e6-4dd1-9ab1-d3999428cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:30.830939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6b0cc06b-e9e6-4dd1-9ab1-d3999428cc81 None None] MFA Rules not processed for user `f00f343268134dea89e0a7070da14508`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:30.832647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6b0cc06b-e9e6-4dd1-9ab1-d3999428cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:30.859232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 834/1638] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:30.860809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:31.069526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9deb-7aa7-49ed-817a-5e7467527216 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:31.116724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:31.119197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:31.119490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:31.119777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:31.149095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:31.161192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb348b0-6716-4fe7-8561-a00bcf9cc674 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:31.480022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 805/1639] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:16:31.867724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad1f8-a29f-4c6c-aff3-3a019a6b0ab6 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:31.868996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad1f8-a29f-4c6c-aff3-3a019a6b0ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:31.869448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad1f8-a29f-4c6c-aff3-3a019a6b0ab6 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:31.871923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 835/1640] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 10 14:16:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:31.879444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5d32-df12-4931-ad23-422ebd54b36b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:31.885434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5d32-df12-4931-ad23-422ebd54b36b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:31.885434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5d32-df12-4931-ad23-422ebd54b36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:31.928793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e46a5d32-df12-4931-ad23-422ebd54b36b None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:31.930646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e46a5d32-df12-4931-ad23-422ebd54b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:32.054086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 806/1641] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:32.085919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad1f8-a29f-4c6c-aff3-3a019a6b0ab6 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:32.117973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad1f8-a29f-4c6c-aff3-3a019a6b0ab6 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:32.125350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:32.126313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:32.126553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:32.126833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:32.148694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:32.150157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fe21e-065d-4360-8b1f-e4c9ffb9423b service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:32.169150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 836/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:33.273200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981776-bf84-4c97-b565-ca43204eb432 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:33.277356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981776-bf84-4c97-b565-ca43204eb432 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:33.277356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981776-bf84-4c97-b565-ca43204eb432 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:33.277356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 807/1643] 10.222.0.58 () {54 vars in 960 bytes} [Tue May 10 14:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:33.290969 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f202-cc50-4875-8f1d-544e2b11c293 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:33.291498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f202-cc50-4875-8f1d-544e2b11c293 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:33.291498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f202-cc50-4875-8f1d-544e2b11c293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:33.374646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ef79f202-cc50-4875-8f1d-544e2b11c293 None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:33.376717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ef79f202-cc50-4875-8f1d-544e2b11c293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:33.528861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 837/1644] 10.222.0.58 () {58 vars in 1057 bytes} [Tue May 10 14:16:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:33.541249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62981776-bf84-4c97-b565-ca43204eb432 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:33.573499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62981776-bf84-4c97-b565-ca43204eb432 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:33.581498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:33.582819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:33.583455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:33.583931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:33.586435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:33.604663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:33.606652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaed12-235f-4879-b6bb-78d6afe25c64 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:33.656156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 808/1645] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:35.521407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:35.521928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:35.522300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:35.588748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] MFA Rules not processed for user `5b4b18c85fae4ea48563786cd11e7c94`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:35.591906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:35.772973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 838/1646] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:37.890633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811358-1dac-41af-983d-07b34cbf72dc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:37.891223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811358-1dac-41af-983d-07b34cbf72dc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:37.891223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86811358-1dac-41af-983d-07b34cbf72dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:37.952305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-86811358-1dac-41af-983d-07b34cbf72dc None None] MFA Rules not processed for user `9dae4d7a57f343c8bb7b92dbebea181d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:37.954006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-86811358-1dac-41af-983d-07b34cbf72dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:38.137964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 809/1647] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:38.158914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:38.181546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e5604-a3b7-4d90-8d10-c126f3e45568 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:38.195948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:38.196952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:38.197422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:38.197907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:38.219132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:38.224066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd74388-d02d-42d0-a43b-a1af680a31f1 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:38.249956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 839/1648] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:40.057353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:40.057759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:40.057915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:40.101059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] MFA Rules not processed for user `ecb4c97543e1452d8e6c8f1564addc76`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:40.102303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:40.402632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:40.406602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:40.406602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:40.478708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:40.479045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 810/1649] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:40.479045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:40.479164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:40.777468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 840/1650] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:16:40.779290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:40.790486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac0a25-1ca7-48bb-864e-406131b3792c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:40.823345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:40.824255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:40.824543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:40.824821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] PATH_INFO: `/v3/users/2186765b0174480881be647fc9b5c3f4` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:40.826183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2186765b0174480881be647fc9b5c3f4)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:40.827260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: WARNING py.warnings [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:40.827260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: warnings.warn(msg) May 10 14:16:40.827260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: May 10 14:16:40.828517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:40.833494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f488e-a67c-433b-b778-0dcba961188f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:40.855005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:40.858078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:40.858609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:40.859211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:40.886598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:40.888570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:40.937617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 841/1651] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:41.221809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2186765b0174480881be647fc9b5c3f4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:41.224554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2186765b0174480881be647fc9b5c3f4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:41.340341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2186765b0174480881be647fc9b5c3f4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:41.343143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2186765b0174480881be647fc9b5c3f4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:41.407650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:41.409500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2186765b0174480881be647fc9b5c3f4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:41.411405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:41.543257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 811/1652] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:40 2022] DELETE /identity/v3/users/2186765b0174480881be647fc9b5c3f4 => generated 0 bytes in 1067 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:41.736698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17400-49dd-484b-9c8b-1c6926ff2857 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:42.080095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:42.081071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:42.081393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:42.081682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:42.084945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:42.277925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:42.280263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61a86-e6a1-434a-bb5b-410506550c24 service swift] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:42.300584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 842/1653] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:16:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 895 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 14:16:43.312845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:43.316512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:43.577611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444b618-2f25-48dc-a40d-afe9dc23ab93 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:43.588997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:43.590268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:43.590863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:43.591329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] PATH_INFO: `/v3/users/0fc0746bf323456394119f58a2ad28fa` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:43.595982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc0746bf323456394119f58a2ad28fa)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:43.602361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:43.909742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc0746bf323456394119f58a2ad28fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:43.918041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc0746bf323456394119f58a2ad28fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:44.000056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc0746bf323456394119f58a2ad28fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:44.003055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc0746bf323456394119f58a2ad28fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:44.020794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-86031493-8fd2-4576-8042-06eb5da37e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc0746bf323456394119f58a2ad28fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:44.333542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe916c-0e99-4ae5-ba1b-f2a5180d67ae None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:44.333542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe916c-0e99-4ae5-ba1b-f2a5180d67ae None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:44.333542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe916c-0e99-4ae5-ba1b-f2a5180d67ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:44.337134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 812/1654] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:43 2022] DELETE /identity/v3/users/0fc0746bf323456394119f58a2ad28fa => generated 0 bytes in 1026 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:44.348056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:44.349427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:44.350925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:44.402531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-97fe916c-0e99-4ae5-ba1b-f2a5180d67ae None None] MFA Rules not processed for user `5e3edc87e7014d8598ba8c9517b81cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:44.405057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-97fe916c-0e99-4ae5-ba1b-f2a5180d67ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:44.407873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:44.410578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:44.607501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 813/1655] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:44.608593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:44.610080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:44.638183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ec364-1f02-45e8-b427-d5893246e8e4 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:44.648141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:44.654274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 843/1656] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:44.655895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:44.656336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:44.657010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] PATH_INFO: `/v3/projects/8d853dcac20846ed9e36e4c21b8bee20` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:44.658885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d853dcac20846ed9e36e4c21b8bee20)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:44.664186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:44.664752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:44.665187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:44.669524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:44.740850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] MFA Rules not processed for user `5e3edc87e7014d8598ba8c9517b81cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:44.746067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:44.786449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 844/1657] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:44.787472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:44.792192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:44.935055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d853dcac20846ed9e36e4c21b8bee20', 'request_id': 'req-d8b58f12-8266-41aa-b15e-d368f434831a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:44.962402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b350bb-a7da-49ba-ae1a-353bd8ab5556 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:45.048274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d853dcac20846ed9e36e4c21b8bee20', 'request_id': 'req-d8b58f12-8266-41aa-b15e-d368f434831a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.055434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d853dcac20846ed9e36e4c21b8bee20', 'request_id': 'req-d8b58f12-8266-41aa-b15e-d368f434831a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.091056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] The token cache is being invalidate because project 8d853dcac20846ed9e36e4c21b8bee20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:16:45.091414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.098755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.116791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 814/1658] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:44 2022] DELETE /identity/v3/projects/8d853dcac20846ed9e36e4c21b8bee20 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:45.119461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:45.123839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:45.163904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:45.165296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:45.165612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:45.166035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:45.170579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:45.343888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:45.348737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:45.371133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 845/1659] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:45.381336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:45.385025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b58f12-8266-41aa-b15e-d368f434831a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:45.388575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:45.396867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:45.398977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:45.401226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:45.401226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:45.404173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:45.420785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2010a5-b61b-463f-8e68-11beb8e54865 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:45.436845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:45.440760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:45.442298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:45.444029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] PATH_INFO: `/v3/users/5b4b18c85fae4ea48563786cd11e7c94` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:45.447964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4b18c85fae4ea48563786cd11e7c94)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:45.454848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:45.542687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:45.544830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209aab4-42a8-4739-afe1-86392dca6713 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:45.555977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 815/1660] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:45.761052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4b18c85fae4ea48563786cd11e7c94'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.761052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4b18c85fae4ea48563786cd11e7c94'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.793278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4b18c85fae4ea48563786cd11e7c94'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.816757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4b18c85fae4ea48563786cd11e7c94'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.817952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4b18c85fae4ea48563786cd11e7c94'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:45.849204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:45.849204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:45.849204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:45.914103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:45.916862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 846/1661] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:45 2022] DELETE /identity/v3/users/5b4b18c85fae4ea48563786cd11e7c94 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:45.921304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:46.524741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 816/1662] 10.222.0.64 () {58 vars in 1015 bytes} [Tue May 10 14:16:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 678 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:16:46.554560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:46.556083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:46.578291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef209b57-777c-4459-9041-ee94b29c5eec admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:46.589185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:46.590843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:46.606683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:46.611934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:46.617667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:46.618156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:46.657632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:46.659173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3104c-751d-4ecc-9c9b-19964f3585da service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:46.685719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 847/1663] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:46.821136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92962182-42e0-427c-a9c6-bb8363f6d169 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:46.847166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:46.849914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:46.850439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:46.850871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] PATH_INFO: `/v3/users/3ee7d0ba919e4ef8bfc874c366d08595` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:46.854862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee7d0ba919e4ef8bfc874c366d08595)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:46.857155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:47.114163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee7d0ba919e4ef8bfc874c366d08595'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:47.117906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee7d0ba919e4ef8bfc874c366d08595'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:47.145432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee7d0ba919e4ef8bfc874c366d08595'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:47.148726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee7d0ba919e4ef8bfc874c366d08595'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:47.162385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4d2ec0f-adaa-4684-a7c3-19effa52adb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee7d0ba919e4ef8bfc874c366d08595'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:47.258008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 817/1664] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:46 2022] DELETE /identity/v3/users/3ee7d0ba919e4ef8bfc874c366d08595 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:47.262584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:47.263069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:47.263563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:47.297213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:47.299705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:47.516933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:47.519189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:47.519189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:47.580670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] MFA Rules not processed for user `8d788b9560504c3795dbb623f10467cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:47.585671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 848/1665] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:47.589812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:47.807651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 818/1666] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:47.843852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:47.846678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:47.933746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:47.935412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:47.984239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447ebcd-118e-47bd-89aa-616c50ed2a54 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:47.990375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2b8f4-a42f-431f-9f05-a9ea5ea54fd9 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:48.004926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:48.005838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.006142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:48.006346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.006838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.007245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.009329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.009700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] PATH_INFO: `/v3/projects/62b05d305fd145fa8d69ca029f9902b8` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.020337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b05d305fd145fa8d69ca029f9902b8)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:48.026070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:48.033690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:48.040895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03e2f9-b290-41ed-8d40-8d494ca28187 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:48.070049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 849/1667] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:48.253854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b05d305fd145fa8d69ca029f9902b8', 'request_id': 'req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:48.323720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b05d305fd145fa8d69ca029f9902b8', 'request_id': 'req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:48.335925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b05d305fd145fa8d69ca029f9902b8', 'request_id': 'req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:48.377798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] The token cache is being invalidate because project 62b05d305fd145fa8d69ca029f9902b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:16:48.379261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:48.380460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1305b350-8033-4fbf-9b92-2eedd8c0fdcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:48.386237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 819/1668] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:47 2022] DELETE /identity/v3/projects/62b05d305fd145fa8d69ca029f9902b8 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:16:48.391973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.392633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.393165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.439875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:48.466171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:48.602279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 850/1669] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:48.603024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.603024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.603523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.640071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:48.645006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:48.681193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:48.695740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:48.697775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be148754-5173-45fa-a33f-ece00ebc04cb None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:48.739976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:48.745200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.746121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.747604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] PATH_INFO: `/v3/projects/1873a08af77f4a998cf5019df662c2f2` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.756350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1873a08af77f4a998cf5019df662c2f2)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:48.758272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: WARNING py.warnings [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 10 14:16:48.758272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: warnings.warn(msg) May 10 14:16:48.758272 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: May 10 14:16:48.759661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:48.772744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 820/1670] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:16:48.780055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:48.826605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc3638-d925-4aa9-8ad2-42e0ddb4625b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:48.854373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:48.855382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:48.856395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:48.856901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:48.858016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:48.858546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:48.861134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:48.911748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 821/1671] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:48.920371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:48.955059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb0b8d-b8c7-4003-8620-898fb03e0560 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:48.987630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1873a08af77f4a998cf5019df662c2f2', 'request_id': 'req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:49.059891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1873a08af77f4a998cf5019df662c2f2', 'request_id': 'req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:49.074035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1873a08af77f4a998cf5019df662c2f2', 'request_id': 'req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:49.164876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] The token cache is being invalidate because project 1873a08af77f4a998cf5019df662c2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:16:49.165189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:49.168768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:49.169555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:49.169902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:49.170610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:49.174968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:49.175755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:49.177916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1b39d-2d84-4c7f-b7f6-cbae77011fb5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:49.179622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 851/1672] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:48 2022] DELETE /identity/v3/projects/1873a08af77f4a998cf5019df662c2f2 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:16:49.180594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:49.182217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:49.336567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 822/1673] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:48 2022] POST /identity/v3/projects => generated 381 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:49.341081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:49.341081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:49.341081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:49.398822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9896-1cb1-4c72-8d9f-abe5d13d7ad3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:49.440869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:49.447610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:49.693757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:49.697239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:49.699480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:49.702913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:49.710920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:49.783775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 823/1674] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:49.785418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:49.802525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:49.848828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e28d-2286-4f13-8b61-5974e8e40cc3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:49.865191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:49.866270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:49.866759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:49.867261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:49.868183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:49.870394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:49.945418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:49.950403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f15a3-a6c6-4b42-93d7-0114ed77304b service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:49.980307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 852/1675] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:49.982613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:49.982915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:49.985928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.019168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 824/1676] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:49 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:50.023637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.067613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58decede-45d3-4747-b55b-1b99fe422e4c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.080618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:50.081904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:50.084939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.113986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.115573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.117501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.118699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.121970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.124841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 853/1677] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:50.135973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.142244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 825/1678] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:50.146154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.163485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc87bf-bc14-453f-9de1-186933da8828 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.184192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.190642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.191411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c8dc-9834-48e7-a7ed-e351f1d7028b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.200352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.201427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.201796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.202325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.204242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.206380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.238426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 854/1679] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:16:50.241806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.269940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 826/1680] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:50.272028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dabc8-a2dc-4fff-9476-2bf33f883156 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.274934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.287956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.288907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.289327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.289707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.291555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.295323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.329632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7069a2f-5128-4ca3-bfb9-c47587563d3f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.392892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.393612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.394288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.394953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] PATH_INFO: `/v3/projects/e1989d8c870646fe94d2358e0cedf151/users/0da9385ae3ff4d3f9674b24de8f0d341/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.403420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 855/1681] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:50 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:50.405032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.468053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1989d8c870646fe94d2358e0cedf151, user_id=0da9385ae3ff4d3f9674b24de8f0d341, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.500974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0a1f-901b-4090-bbba-370807b281f4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.572349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1249ed6-48db-466c-89db-46e85860e980 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:50.592998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:50.593411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 827/1682] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:50 2022] PUT /identity/v3/projects/e1989d8c870646fe94d2358e0cedf151/users/0da9385ae3ff4d3f9674b24de8f0d341/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:50.604047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.604047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.604047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.604047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:50.607134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:50.607220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:50.609057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:50.609363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:50.695885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] MFA Rules not processed for user `0da9385ae3ff4d3f9674b24de8f0d341`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:50.698445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:50.908718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 856/1683] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:50 2022] POST /identity/v3/users => generated 283 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:50.911426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 828/1684] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:50.914114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.915953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:50.947280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0ebc6-d050-40b8-8942-f9ad452709d9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.109203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bfae2-9fa0-4778-bb4d-f7b8beb26d0c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.121766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.122736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.123168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.124021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.131564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.133884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.151753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.152726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.154409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.154409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.154595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.156789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97dc9d-636d-410e-aeef-4e7f2add8d99 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.180320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 857/1685] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:51.183876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.185710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.186168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.242338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] MFA Rules not processed for user `5e3edc87e7014d8598ba8c9517b81cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:51.245097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:51.347786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 829/1686] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:50 2022] POST /identity/v3/projects => generated 381 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:51.353635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:51.414631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 858/1687] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:51.415676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:51.433038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91c23b-433f-4ea7-846b-6807b22cdaf3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.534631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.535299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.535735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.536031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.536897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.539019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.672261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46db954-3ae2-4928-8bbc-97f370454d3b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.678584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.679261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 859/1688] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:51 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:51.679695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.680229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.684789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:51.685955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.687393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.689935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.693860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:51.726661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 830/1689] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:51.729841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:51.743703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c33b-8027-4b2e-a99e-2afe798abc20 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.750708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.751980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.752307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.752550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.753254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.754925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.759755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502f82d-b39f-4478-9f5f-8c94f7b89e41 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.769518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 831/1690] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:51.770610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:51.773539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.777053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] PATH_INFO: `/v3/projects/3ca7cd9d55b744c88d56d22cb74f4075` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.778845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca7cd9d55b744c88d56d22cb74f4075)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.783398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c2b0-ca95-4da2-b0d8-6fc9c55c656e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:51.790782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:51.791781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:51.792073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:51.792355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] PATH_INFO: `/v3/projects/4d5a2a0a0a1449d381921b1d0f8eb29c/users/d1bff9d8fbaf40e3b9a85595f776ef27/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:51.793850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.817702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5a2a0a0a1449d381921b1d0f8eb29c, user_id=d1bff9d8fbaf40e3b9a85595f776ef27, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:51.820970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:51.983136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 832/1691] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:51 2022] PUT /identity/v3/projects/4d5a2a0a0a1449d381921b1d0f8eb29c/users/d1bff9d8fbaf40e3b9a85595f776ef27/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:51.992914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:52.032700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca7cd9d55b744c88d56d22cb74f4075', 'request_id': 'req-469e43c1-5715-47ca-921e-9c258d9d924d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:52.099227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca7cd9d55b744c88d56d22cb74f4075', 'request_id': 'req-469e43c1-5715-47ca-921e-9c258d9d924d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:52.106543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca7cd9d55b744c88d56d22cb74f4075', 'request_id': 'req-469e43c1-5715-47ca-921e-9c258d9d924d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:52.134570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] The token cache is being invalidate because project 3ca7cd9d55b744c88d56d22cb74f4075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:16:52.137247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:52.141086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-469e43c1-5715-47ca-921e-9c258d9d924d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:52.150111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 860/1692] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:51 2022] DELETE /identity/v3/projects/3ca7cd9d55b744c88d56d22cb74f4075 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:52.150111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90985-fb1a-42fb-9a3d-66e6df001d7c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.150111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90985-fb1a-42fb-9a3d-66e6df001d7c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.150111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90985-fb1a-42fb-9a3d-66e6df001d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.234946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ccc90985-fb1a-42fb-9a3d-66e6df001d7c None None] MFA Rules not processed for user `9dae4d7a57f343c8bb7b92dbebea181d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:52.236275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ccc90985-fb1a-42fb-9a3d-66e6df001d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.279524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57df2f-fc5d-4bd8-8d03-71435ae9ced1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:52.427439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 861/1693] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:52.428595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.428883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.429156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.455277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:52.463258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.463772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.464125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] PATH_INFO: `/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14/users/0dd49a8605cd44d79a53ce79e4cbb28d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.500512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] MFA Rules not processed for user `d1bff9d8fbaf40e3b9a85595f776ef27`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:52.502033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bbcdeb7c8947ed8d37beb43205dc14, user_id=0dd49a8605cd44d79a53ce79e4cbb28d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:52.506115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d842-8e93-404c-be60-40e8ca8e14ed admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:52.510833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.603572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 833/1694] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:51 2022] PUT /identity/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14/users/0dd49a8605cd44d79a53ce79e4cbb28d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:52.604812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa05b2-dea4-4753-ad07-d1a3e187947d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.605176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa05b2-dea4-4753-ad07-d1a3e187947d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.605662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa05b2-dea4-4753-ad07-d1a3e187947d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.609411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 834/1695] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:16:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:52.611469 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368338f-c3ee-4405-871b-ec93413c4f79 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.613820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368338f-c3ee-4405-871b-ec93413c4f79 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.615820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368338f-c3ee-4405-871b-ec93413c4f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.658519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c368338f-c3ee-4405-871b-ec93413c4f79 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:52.660852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c368338f-c3ee-4405-871b-ec93413c4f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.663231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 862/1696] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:52.664502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:52.666339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.698500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174175ba-4ff8-4bc6-910a-d4fdc7dbc491 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:52.864587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 835/1697] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:52.867418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb401-dd83-4480-8923-64ec7e8ddbcb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.867744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:52.868868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.869422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.869867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.870352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb401-dd83-4480-8923-64ec7e8ddbcb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.872526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb401-dd83-4480-8923-64ec7e8ddbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.873378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:52.879535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:52.903648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 863/1698] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:52.905075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:52.910223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.918789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d7afb401-dd83-4480-8923-64ec7e8ddbcb None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:52.921877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d7afb401-dd83-4480-8923-64ec7e8ddbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:52.938407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61651222-0e56-4b9e-81b0-7011964920c8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:52.956939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:52.957631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:52.957919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:52.958229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:52.959004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:52.961200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.048870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 836/1699] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:16:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:53.051898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.052351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.052777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.099353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 864/1700] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:52 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:53.100299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.106518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:53.114884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:53.336712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f967230-0e38-46eb-b937-cee92ed79595 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.347591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 837/1701] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:53.354755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.355764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.356696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.356978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.357257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] PATH_INFO: `/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14/users/0dd49a8605cd44d79a53ce79e4cbb28d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.361510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bbcdeb7c8947ed8d37beb43205dc14, user_id=0dd49a8605cd44d79a53ce79e4cbb28d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.364222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.371332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e3e54-b3e2-4ee9-bc68-b652b91ad29f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.393722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 865/1702] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:53 2022] PUT /identity/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14/users/0dd49a8605cd44d79a53ce79e4cbb28d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:53.407298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.525155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.538573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.539399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.539885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.540790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.543415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.585749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cceaac-60d7-4c42-a0d2-af6760e71e4d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.598141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.600829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.601582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.602154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.603655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:16:53.604228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.609297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ac4ba-0603-4bc1-b69c-b8215ea32485 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.659735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 866/1703] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:16:53.666915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.667433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 838/1704] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:53 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:53.668598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.669167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.669595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.671044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:53.718302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] MFA Rules not processed for user `0dd49a8605cd44d79a53ce79e4cbb28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:53.720230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:53.883127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c3b80-327a-4f41-ab27-90e8b097a53e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.886124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 867/1705] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:53.892675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.897605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.898535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.901864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.902435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.925497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.927537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.930749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6ba4a-6b95-4b9c-a7eb-fb58332a861f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.944301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.945022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.945310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.945591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.946319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.953640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b10adb-e36d-4d67-8331-e7e5b8a43d17 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:53.962785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 839/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:53.964158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:53.980906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220189a4-a958-4e9e-b667-4d751d4dd752 service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:53.986834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:53.987816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:53.988285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:53.988758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:53.989708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:53.991934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:54.017300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 840/1707] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:54.018582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:54.038549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:54.098873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 868/1708] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:16:53 2022] POST /identity/v3/projects => generated 383 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:54.107095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d536ea-ecc4-478e-875c-d1f8264a677a None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.107738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d536ea-ecc4-478e-875c-d1f8264a677a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.107738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d536ea-ecc4-478e-875c-d1f8264a677a None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.116134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 869/1709] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 10 14:16:54 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:16:54.121353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d536ea-ecc4-478e-875c-d1f8264a677a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:54.249141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f501224-fa3c-43b6-a931-53e0c92addb5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:54.336106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:54.338728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.339201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.341963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.365612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d536ea-ecc4-478e-875c-d1f8264a677a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:54.372777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:54.374372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.374701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.374934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] PATH_INFO: `/v3/projects/67801a189cdf42179ed272ab6444f1ee/users/e3d8a5f848ac468faa89ed05e55d9bf6/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.450109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67801a189cdf42179ed272ab6444f1ee, user_id=e3d8a5f848ac468faa89ed05e55d9bf6, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:54.467487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:54.585788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 870/1710] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:54 2022] PUT /identity/v3/projects/67801a189cdf42179ed272ab6444f1ee/users/e3d8a5f848ac468faa89ed05e55d9bf6/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:54.586439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:54.600368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:54.606173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70277f54-151f-4e12-a200-5947c58ec139 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:54.635394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 841/1711] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:54.645645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.647293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.647293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.709024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:54.714290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:54.784574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ae898-bb64-426e-b706-7601057bbe50 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:54.792735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:54.793525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.794248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.794720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.795625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:54.798646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa226-7d17-4baf-b009-b6c7601b261f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:54.881339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 842/1712] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:16:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:54.882888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:54.941454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1371-ea60-4f79-82ae-aa199eae1236 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:54.953824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 871/1713] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:16:54 2022] POST /identity/v3/users => generated 292 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:16:54.955774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.956784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.957167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.962447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:54.963361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:54.965136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:54.965585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:54.967696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:54.985193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:54.989746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] MFA Rules not processed for user `ecb4c97543e1452d8e6c8f1564addc76`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:54.991569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:55.034580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 843/1714] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:55.036662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.040714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:55.071358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644c4c-fb40-4011-8117-8409484b9820 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.084059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.085317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.085577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.085977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.100921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.102577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.133413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 844/1715] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:55.139725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 872/1716] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:55.140248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.140341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.149482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc08a69-0dc2-4948-9ddb-f33f08a89d77 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.155767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.158270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.158715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.159108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] PATH_INFO: `/v3/projects/67801a189cdf42179ed272ab6444f1ee/users/e3d8a5f848ac468faa89ed05e55d9bf6/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.161830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67801a189cdf42179ed272ab6444f1ee, user_id=e3d8a5f848ac468faa89ed05e55d9bf6, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.165322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.169182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1b294-61f1-4fe7-8361-82f12921b3c2 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.184671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.187468 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.188017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.189511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.190032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 873/1717] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:55 2022] PUT /identity/v3/projects/67801a189cdf42179ed272ab6444f1ee/users/e3d8a5f848ac468faa89ed05e55d9bf6/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:55.192564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.193486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.196586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22e3f5-2969-44ad-be28-8313f0d85c07 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.254347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 845/1718] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:55.258033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.258983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.266766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.359777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] MFA Rules not processed for user `e3d8a5f848ac468faa89ed05e55d9bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:55.363104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:55.381229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621cf9e-51ce-4307-b7e2-b6a5a3b64ed8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.397646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.398942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.400900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.401643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.405527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:55.575655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.577616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.608047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 874/1719] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:55.612609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 846/1720] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:55.613575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.628494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.630546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:55.684276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed885a-e05a-4bdd-89b8-a9f30600d3ab None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.694242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.695049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.697196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.697982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.732900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.736717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.771241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 847/1721] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:16:55.823623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc1f5-939e-45d5-9272-0f9b121df3af service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.851173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.852255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.853711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.855501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.860047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.864469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239743-b631-4de8-988e-8ab09797159a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:55.885101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 875/1722] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:16:55.891345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:55.908773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1b9b7-734d-481f-8615-536247453b72 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:55.916856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:55.919949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:55.920804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:55.922132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] PATH_INFO: `/v3/projects/b6622387ea384e53932f123ee92e695c/users/8c1b3cb0eff54a038e518dc6fac9c8e8/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:55.973726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6622387ea384e53932f123ee92e695c, user_id=8c1b3cb0eff54a038e518dc6fac9c8e8, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:55.982162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:56.114187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 848/1723] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:16:55 2022] PUT /identity/v3/projects/b6622387ea384e53932f123ee92e695c/users/8c1b3cb0eff54a038e518dc6fac9c8e8/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:56.117097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:56.117581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:56.117859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:56.192852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] MFA Rules not processed for user `8c1b3cb0eff54a038e518dc6fac9c8e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:56.195072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:56.504819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 876/1724] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:56.517998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:56.669971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6d7c5-fd6c-4a9e-ae46-cc596509bfa5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:56.676294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:56.677479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:56.677833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:56.678046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:56.694191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:56.695715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2469f52-97e0-4101-acd2-7f50980df608 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:56.714876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 849/1725] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:56.747240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:56.930234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac5fb-8df7-4ecb-9198-b1dcfb1303f2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:56.939482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:56.940569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:56.940861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:56.944330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] PATH_INFO: `/v3/users/8c1b3cb0eff54a038e518dc6fac9c8e8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:56.944330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1b3cb0eff54a038e518dc6fac9c8e8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:56.944983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:57.208457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1b3cb0eff54a038e518dc6fac9c8e8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:57.214700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1b3cb0eff54a038e518dc6fac9c8e8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:57.288927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1b3cb0eff54a038e518dc6fac9c8e8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:57.318717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1b3cb0eff54a038e518dc6fac9c8e8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:57.319852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1b3cb0eff54a038e518dc6fac9c8e8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:57.774300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 877/1726] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:56 2022] DELETE /identity/v3/users/8c1b3cb0eff54a038e518dc6fac9c8e8 => generated 0 bytes in 1028 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:57.779198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:57.780466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:57.780855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:57.832648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:57.832648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:57.838049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:57.839956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:57.937433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 850/1727] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:57.951323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:57.972563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:58.056552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d2d7-0a9d-408c-974f-03b25eb63598 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:58.073908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:58.080960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:58.082570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:58.083823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] PATH_INFO: `/v3/users/d1bff9d8fbaf40e3b9a85595f776ef27` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:58.094014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bff9d8fbaf40e3b9a85595f776ef27)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:58.101563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:58.192984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcb3bd-3e46-4815-a00a-6f62c1d2c226 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:58.231570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:58.231570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:58.231570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:58.231570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:58.255701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:58.271234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee379bd-61cb-4a73-918b-c02550e3fd94 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:58.294068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 851/1728] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:16:58.390489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bff9d8fbaf40e3b9a85595f776ef27'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:58.393795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bff9d8fbaf40e3b9a85595f776ef27'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:58.435301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bff9d8fbaf40e3b9a85595f776ef27'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:58.480674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bff9d8fbaf40e3b9a85595f776ef27'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:58.486149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bff9d8fbaf40e3b9a85595f776ef27'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:58.565955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 878/1729] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:16:57 2022] DELETE /identity/v3/users/d1bff9d8fbaf40e3b9a85595f776ef27 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:58.570454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:58.570788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:58.571069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:58.611744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:58.613813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:58.721117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 852/1730] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:58.739418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:58.740936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:58.836200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e0c52-b84f-4f61-a0fd-675df4f46c23 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:58.851990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:58.853690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:58.854233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:58.854813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:58.879215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:58.887718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce917f-fedd-46aa-a865-c17447debcd8 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:58.919089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 879/1731] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:16:59.371738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:59.373398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:59.405143 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2b99e-108b-4695-aa98-b677d60494ae None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:59.414421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:59.415307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:59.415641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:59.415930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] PATH_INFO: `/v3/projects/b6622387ea384e53932f123ee92e695c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:59.417130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6622387ea384e53932f123ee92e695c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:59.419228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:59.600328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6622387ea384e53932f123ee92e695c', 'request_id': 'req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:59.647554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093be0-e325-43a9-84fc-eb8292ad8439 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:59.648265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093be0-e325-43a9-84fc-eb8292ad8439 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:59.648746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093be0-e325-43a9-84fc-eb8292ad8439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:59.658969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6622387ea384e53932f123ee92e695c', 'request_id': 'req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:59.674049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a5093be0-e325-43a9-84fc-eb8292ad8439 None None] MFA Rules not processed for user `8d788b9560504c3795dbb623f10467cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:59.682803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a5093be0-e325-43a9-84fc-eb8292ad8439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:59.691067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6622387ea384e53932f123ee92e695c', 'request_id': 'req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:59.752332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] The token cache is being invalidate because project b6622387ea384e53932f123ee92e695c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:16:59.752780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:59.753747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:16:59.763415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 853/1732] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:59 2022] DELETE /identity/v3/projects/b6622387ea384e53932f123ee92e695c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:16:59.768080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:59.773539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:59.828442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 880/1733] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:59.831803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11582f-006f-4a6a-a2ad-b9a726eb4227 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:59.832269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11582f-006f-4a6a-a2ad-b9a726eb4227 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:59.832872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11582f-006f-4a6a-a2ad-b9a726eb4227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:59.862471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3dfc1-1bbb-404c-aa33-2fdd3eb5a1d8 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:59.864524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7e11582f-006f-4a6a-a2ad-b9a726eb4227 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:59.866410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7e11582f-006f-4a6a-a2ad-b9a726eb4227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:59.871101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:59.872002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:59.872589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:59.872888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] PATH_INFO: `/v3/projects/4d5a2a0a0a1449d381921b1d0f8eb29c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:59.877028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5a2a0a0a1449d381921b1d0f8eb29c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:16:59.878540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 881/1734] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:59.879091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:16:59.886075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:16:59.886782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:16:59.887599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:16:59.938237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:16:59.961873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:16:59.971899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 882/1735] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:16:59.975820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:16:59.988276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28e46d-7236-42ea-8cfa-b5b4592ebb82 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:16:59.998609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:16:59.999669 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.000663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.002730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.006466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:00.007019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.008678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.039455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 883/1736] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:00.070487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.084085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfebb9-ba0c-4501-abbb-ca87d4758a94 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.106146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5a2a0a0a1449d381921b1d0f8eb29c', 'request_id': 'req-84e5344a-870b-49ae-8f1a-65cd6258cdae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.224915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.226146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.226614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.227056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.227962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.230703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.252619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5a2a0a0a1449d381921b1d0f8eb29c', 'request_id': 'req-84e5344a-870b-49ae-8f1a-65cd6258cdae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.255638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5a2a0a0a1449d381921b1d0f8eb29c', 'request_id': 'req-84e5344a-870b-49ae-8f1a-65cd6258cdae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.271795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] The token cache is being invalidate because project 4d5a2a0a0a1449d381921b1d0f8eb29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:00.272132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.272844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.275000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 854/1737] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:16:59 2022] DELETE /identity/v3/projects/4d5a2a0a0a1449d381921b1d0f8eb29c => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:00.279220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.281455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:00.299966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5344a-870b-49ae-8f1a-65cd6258cdae admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.312027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.320245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.320245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.327425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] PATH_INFO: `/v3/users/e3d8a5f848ac468faa89ed05e55d9bf6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.327425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d8a5f848ac468faa89ed05e55d9bf6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.327425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.378039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 884/1738] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:00 2022] POST /identity/v3/projects => generated 371 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:00.379507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.382258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:00.482122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdd3c2-41df-43ef-ae5d-054fcaad65d7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.501363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.501784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.502356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.502808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.503559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.505654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.591400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d8a5f848ac468faa89ed05e55d9bf6'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.593591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d8a5f848ac468faa89ed05e55d9bf6'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.616732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d8a5f848ac468faa89ed05e55d9bf6'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.617634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d8a5f848ac468faa89ed05e55d9bf6'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.630067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-492e793b-c570-4737-af3e-6fcff65b057d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d8a5f848ac468faa89ed05e55d9bf6'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:00.640501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 885/1739] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:00 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:00.641570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.644170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:00.657894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f029667-9093-4142-ba71-6a0c10e16be6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.664253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.670173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.670173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.670173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.670173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.670173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.680080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 886/1740] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:00.685401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.697138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fc89-9fad-44d6-8362-b655ada8bfeb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.702233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.702984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.703274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.704924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.705733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.708045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.723238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 887/1741] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:00.725955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:00.737761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c1d91-99ef-420d-801c-78135687fc52 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:00.743063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:00.744107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.767845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.767845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] PATH_INFO: `/v3/projects/afe8e2baf88941ad800d81a47a96b538/users/a07b83a18e164c10adefb2c8e5265952/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.769398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe8e2baf88941ad800d81a47a96b538, user_id=a07b83a18e164c10adefb2c8e5265952, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:00.772141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:00.803172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 855/1742] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:00 2022] DELETE /identity/v3/users/e3d8a5f848ac468faa89ed05e55d9bf6 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:00.862423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 888/1743] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:00 2022] PUT /identity/v3/projects/afe8e2baf88941ad800d81a47a96b538/users/a07b83a18e164c10adefb2c8e5265952/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:00.866264 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:00.866717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:00.867159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:00.932368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] MFA Rules not processed for user `a07b83a18e164c10adefb2c8e5265952`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:00.934158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:01.030664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 856/1744] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:01.039618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:01.041465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:01.192297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c996442-6d46-4ec7-8559-c0fdcfabf700 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:01.199151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:01.200026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:01.200452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:01.201374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:01.212982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:01.214665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:01.228417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 889/1745] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:01.263099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:01.424707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb395f-7e53-43b7-8758-b2964ac27976 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:01.440280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:01.441403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:01.441889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:01.442538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] PATH_INFO: `/v3/users/a07b83a18e164c10adefb2c8e5265952` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:01.445620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07b83a18e164c10adefb2c8e5265952)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:01.447696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:01.636594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:01.638685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:01.714243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5821-e552-4711-8305-2f16240e73e9 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:01.727945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07b83a18e164c10adefb2c8e5265952'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.727945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07b83a18e164c10adefb2c8e5265952'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.734498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:01.736007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:01.737126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:01.737126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] PATH_INFO: `/v3/projects/67801a189cdf42179ed272ab6444f1ee` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:01.744241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67801a189cdf42179ed272ab6444f1ee)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:01.755942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:01.756285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07b83a18e164c10adefb2c8e5265952'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.757439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07b83a18e164c10adefb2c8e5265952'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.778964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4acc622-2337-4788-a2e9-c30a014bf94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07b83a18e164c10adefb2c8e5265952'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.856785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 857/1746] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:01 2022] DELETE /identity/v3/users/a07b83a18e164c10adefb2c8e5265952 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:01.858651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:01.862400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:01.862843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:01.879154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67801a189cdf42179ed272ab6444f1ee', 'request_id': 'req-3d2b842a-4227-40b7-b2eb-8350b22011c6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.918390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:01.920347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:01.944710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67801a189cdf42179ed272ab6444f1ee', 'request_id': 'req-3d2b842a-4227-40b7-b2eb-8350b22011c6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.962513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67801a189cdf42179ed272ab6444f1ee', 'request_id': 'req-3d2b842a-4227-40b7-b2eb-8350b22011c6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.977699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] The token cache is being invalidate because project 67801a189cdf42179ed272ab6444f1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:01.978429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.978678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-3d2b842a-4227-40b7-b2eb-8350b22011c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:01.980518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 890/1747] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:01 2022] DELETE /identity/v3/projects/67801a189cdf42179ed272ab6444f1ee => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:01.985128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26817-aa14-4e76-a6a0-fd6b74ead0c0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:01.985967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26817-aa14-4e76-a6a0-fd6b74ead0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:01.986468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa26817-aa14-4e76-a6a0-fd6b74ead0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.004866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 858/1748] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:02.006713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-cfa26817-aa14-4e76-a6a0-fd6b74ead0c0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:02.008421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cfa26817-aa14-4e76-a6a0-fd6b74ead0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:02.014749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.016167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:02.024297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 891/1749] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:02.035238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.035510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.035791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.059270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:02.060765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:02.081422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 892/1750] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:02.085672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.087224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2e749-ea3b-4f42-abb8-626527a93bac None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.090974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.091688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.091957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.092236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.095567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51db77-c7fb-4fcd-8fe0-8abe9e5c9304 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.100850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.101520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.101810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.102091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.103071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:02.103385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.103913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.105005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.116119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.121528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 893/1751] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:02.125758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.134588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 859/1752] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:02.138295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebae30-fbe0-42b9-a95d-38bc53070b5a admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.144965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.146265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.146859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.147289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.148196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.150890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.273393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 894/1753] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:02 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:02.277833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.287966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a131e25-5f52-4ca3-817f-5392dff803c6 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.346698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.347363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.347771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.348054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.348751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.350897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.427292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 860/1754] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:02 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:02.431621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.442758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda91ce5-4460-4216-9849-03dc1b9ab3cc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.447636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.448305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.448587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.448865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.449591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.451221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.461481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 895/1755] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:02.465680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.476830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf97ac-d691-4f4a-9b68-7120a5d74ae7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.482399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.483096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.483388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.483646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.484397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.486679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.497869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 861/1756] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:02.502425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.513706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334f0e9-ed0f-4177-956e-da21dc4551cb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.518671 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.519560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.519846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.520119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] PATH_INFO: `/v3/projects/03c81d7d451f441d8fd192c8c3b7ab8c/users/ac6209313bfd4650b3651a9347417c8c/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.538148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c81d7d451f441d8fd192c8c3b7ab8c, user_id=ac6209313bfd4650b3651a9347417c8c, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.538355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.539926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:02.540867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9aa27-e22f-4acc-9666-cd26f59a24b7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.555126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6b426-e752-4d15-8e99-ef6c0267f921 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.568194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.569324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.569616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.570545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] PATH_INFO: `/v3/projects/afe8e2baf88941ad800d81a47a96b538` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.571759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe8e2baf88941ad800d81a47a96b538)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.574384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:02.619237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 896/1757] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:02 2022] PUT /identity/v3/projects/03c81d7d451f441d8fd192c8c3b7ab8c/users/ac6209313bfd4650b3651a9347417c8c/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:02.630182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.630680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.647074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.660166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] MFA Rules not processed for user `ac6209313bfd4650b3651a9347417c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:02.661383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:02.761760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 897/1758] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:02.763556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:02.770696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe8e2baf88941ad800d81a47a96b538', 'request_id': 'req-02fad00e-f6de-45bc-83ea-766918a1b5ae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:02.839631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09859623-b820-46ac-8adc-e4768e9600e8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:02.898846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:02.899699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:02.899968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:02.900274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:02.901111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:02.903136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.597010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe8e2baf88941ad800d81a47a96b538', 'request_id': 'req-02fad00e-f6de-45bc-83ea-766918a1b5ae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:03.603112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe8e2baf88941ad800d81a47a96b538', 'request_id': 'req-02fad00e-f6de-45bc-83ea-766918a1b5ae'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:03.618853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] The token cache is being invalidate because project afe8e2baf88941ad800d81a47a96b538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:03.619157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:03.619994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-02fad00e-f6de-45bc-83ea-766918a1b5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:03.622056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 862/1759] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:02 2022] DELETE /identity/v3/projects/afe8e2baf88941ad800d81a47a96b538 => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:03.627760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb7e50-4fc8-403f-a17e-3248e55686db None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.634752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 898/1760] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:02 2022] POST /identity/v3/projects => generated 369 bytes in 869 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:03.634892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb7e50-4fc8-403f-a17e-3248e55686db None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.634892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb7e50-4fc8-403f-a17e-3248e55686db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.636458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:03.638486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:03.656460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6bbb7e50-4fc8-403f-a17e-3248e55686db None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:03.664854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6bbb7e50-4fc8-403f-a17e-3248e55686db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:03.739697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22ef00-4e09-4415-9a52-304f1b8bbfa4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:03.739835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 863/1761] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:03.739919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:03.739919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.739919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.739919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.740195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:03.740195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.745928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.746245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.746525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.765108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:03.766372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:03.774657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 864/1762] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:03.778833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:03.787702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042bed1-fca5-4407-bb65-91ba837bb287 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:03.792311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:03.792939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.793219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.793493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.794227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:03.794533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:03.796251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.808056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 865/1763] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:17:03.814113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:03.822090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd5f1-8f9e-4ef9-9ac2-0caa5d10f9ee admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:03.826690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:03.827291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.827573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.827848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.828510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:03.830490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.871121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 899/1764] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:03 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:03.874823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:03.884941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811940-f611-4b95-af74-9b262d58cee0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:03.889568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:03.890222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.890532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.890829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.891574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:03.893177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.902786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 900/1765] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:03.914074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:03.924544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fef28-c7bd-4b0e-a023-f6deb36975e6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:03.929116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:03.929985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:03.930320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:03.930601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] PATH_INFO: `/v3/projects/3bf611fe134c45d18022145f1ed541d7/users/c1373425b33d448ba86661d6f21f4335/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:03.946678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf611fe134c45d18022145f1ed541d7, user_id=c1373425b33d448ba86661d6f21f4335, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:03.949479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:03.969978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 866/1766] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:03 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:03.973997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.038896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdd5e3-edc0-472e-bb72-299c86ff6aa9 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.043765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.044453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.044753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.045006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.045729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.047735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.055982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 901/1767] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:03 2022] PUT /identity/v3/projects/3bf611fe134c45d18022145f1ed541d7/users/c1373425b33d448ba86661d6f21f4335/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:04.059929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.130663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377805-56d8-47f5-a49d-53411529f15b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.135184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 867/1768] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:03 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:04.138752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.139026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.140799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.140913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.143371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.143371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.143996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.150398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aff9-5943-40ea-b817-14cd8a025124 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.155642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.156392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.156749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.157637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.158482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.160159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.161806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 902/1769] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:04.166066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.171505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 868/1770] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:04.176703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.177650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02e9e5-c67f-4f18-8ab8-7d8f9d4bcdad admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.186574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c642bb-3a4b-4c15-86ca-5aa150ae4dda admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.187483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.188826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.189379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.189691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] PATH_INFO: `/v3/projects/3bf611fe134c45d18022145f1ed541d7/users/c1373425b33d448ba86661d6f21f4335/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.191760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf611fe134c45d18022145f1ed541d7, user_id=c1373425b33d448ba86661d6f21f4335, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.191862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.192761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.192943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.193337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.194128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.195600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e41bb-70a5-4d0b-b34b-82636737ae7d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.196027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.211638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 869/1772] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:04.215808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.218508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 903/1772] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:04 2022] PUT /identity/v3/projects/3bf611fe134c45d18022145f1ed541d7/users/c1373425b33d448ba86661d6f21f4335/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:04.222239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.222798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.223288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.232748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096eeb9-6607-47bb-a2e5-27332483fc97 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.254414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] MFA Rules not processed for user `c1373425b33d448ba86661d6f21f4335`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:04.255944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.310692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.312442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.313031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.313820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] PATH_INFO: `/v3/projects/2c1037d775114df4b21dbdfa5f639cb6/users/7669f2f83550428893d6c91614b78f76/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.329601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1037d775114df4b21dbdfa5f639cb6, user_id=7669f2f83550428893d6c91614b78f76, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.332528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df49e3-f08a-47ae-a652-6f17ada2bee5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.339088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 904/1773] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:04.351436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.352933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.409567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3de8-b897-4583-9850-f857f900f748 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.413147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.413841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.414067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.414372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.420501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 870/1774] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:04 2022] PUT /identity/v3/projects/2c1037d775114df4b21dbdfa5f639cb6/users/7669f2f83550428893d6c91614b78f76/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:04.424001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.425425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.426294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.426648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.426927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.435103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 905/1775] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:04.444095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:04.445289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.503870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 871/1776] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:04.513707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.515145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.540257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:04.541655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.573195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411351-b178-4a2c-9ce6-4763fdf1cc35 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.599480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e7fa1-6359-4703-b05f-77880db0ad15 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:04.605086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.605798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.606091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.606393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.608712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:04.622919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:04.623583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:04.623866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:04.624416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:04.634324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.635763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc272e3-ff63-4957-b4db-63791f8a5ec7 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.646709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 906/1777] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:04.671864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:04.673864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:04.683078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 872/1778] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:06.303518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abf6c-ad8f-4233-a939-aac19bc64934 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:06.303896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abf6c-ad8f-4233-a939-aac19bc64934 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:06.304881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abf6c-ad8f-4233-a939-aac19bc64934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:06.345949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b54abf6c-ad8f-4233-a939-aac19bc64934 None None] MFA Rules not processed for user `7669f2f83550428893d6c91614b78f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:06.347337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b54abf6c-ad8f-4233-a939-aac19bc64934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:06.441078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 907/1779] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:06.457865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:06.476974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68f5ff-a240-4766-80e2-1e4274a7f225 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:06.491179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:06.491833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:06.492119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:06.492396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:06.533156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:06.534670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ba254-0269-4427-b53b-a02cb57ba9d8 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:06.544200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 873/1780] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:08.293966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:08.297227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:08.297227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:08.297227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 908/1781] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:17:08.311091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c368-a6a5-451d-9e2d-2d2dc8039550 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:08.312403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c368-a6a5-451d-9e2d-2d2dc8039550 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:08.312891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c368-a6a5-451d-9e2d-2d2dc8039550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:08.342165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8471c368-a6a5-451d-9e2d-2d2dc8039550 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:08.343728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8471c368-a6a5-451d-9e2d-2d2dc8039550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:08.499399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 874/1782] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:08.592797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:08.597116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:08.684481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d45c24-157a-4392-af14-f78f72241463 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:08.693163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:08.694057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:08.694508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:08.694919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:08.713402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:08.716473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:08.740298 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 909/1783] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:09.509037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c335-7778-49db-a2d0-a17876d35dc3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:09.509479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c335-7778-49db-a2d0-a17876d35dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:09.509598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c335-7778-49db-a2d0-a17876d35dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:09.531642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b800c335-7778-49db-a2d0-a17876d35dc3 None None] MFA Rules not processed for user `7669f2f83550428893d6c91614b78f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:09.534593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b800c335-7778-49db-a2d0-a17876d35dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:09.557184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 875/1784] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:10.473179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:10.493044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0e61e-d09a-4be3-829a-62e1e7ae3fdc service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:10.523604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:10.524219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:10.524771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:10.525366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] PATH_INFO: `/v3/users/ac6209313bfd4650b3651a9347417c8c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:10.526912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6209313bfd4650b3651a9347417c8c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:10.529065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:10.789652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6209313bfd4650b3651a9347417c8c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:10.791744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6209313bfd4650b3651a9347417c8c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:10.809311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6209313bfd4650b3651a9347417c8c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:10.819283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6209313bfd4650b3651a9347417c8c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:10.820659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6209313bfd4650b3651a9347417c8c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:10.913293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 910/1785] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:10 2022] DELETE /identity/v3/users/ac6209313bfd4650b3651a9347417c8c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:10.919635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:10.920129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:10.920216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:10.953789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:10.956373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:11.039452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 876/1786] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:11.051307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:11.053242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:11.125163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c678b2f-ccbe-47c1-818c-2bccec56d1dc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:11.132411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:11.133182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:11.133363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:11.133640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:11.152588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:11.155609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:11.185642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 911/1787] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:11.754313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:11.755793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:11.768044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07782d-5d9d-4bea-9e96-fc3aff520271 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:11.772575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:11.773429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:11.773709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:11.773984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] PATH_INFO: `/v3/users/0da9385ae3ff4d3f9674b24de8f0d341` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:11.775211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da9385ae3ff4d3f9674b24de8f0d341)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:11.777142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:11.790248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:11.792184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:11.810510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6dc81-ba47-45c2-82f1-730abdcf58b5 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:11.818832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:11.819914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:11.820271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:11.820612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] PATH_INFO: `/v3/projects/03c81d7d451f441d8fd192c8c3b7ab8c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:11.822038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c81d7d451f441d8fd192c8c3b7ab8c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:11.824090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:11.969051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da9385ae3ff4d3f9674b24de8f0d341'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:11.970232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c81d7d451f441d8fd192c8c3b7ab8c', 'request_id': 'req-04774d22-e1b1-4d81-9200-8d858a45a51e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:11.970424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da9385ae3ff4d3f9674b24de8f0d341'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:11.984644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da9385ae3ff4d3f9674b24de8f0d341'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:11.985015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da9385ae3ff4d3f9674b24de8f0d341'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:11.992969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4b22bcc0-200e-481c-86ec-a32ef3ed3e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da9385ae3ff4d3f9674b24de8f0d341'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.054735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c81d7d451f441d8fd192c8c3b7ab8c', 'request_id': 'req-04774d22-e1b1-4d81-9200-8d858a45a51e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.060803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c81d7d451f441d8fd192c8c3b7ab8c', 'request_id': 'req-04774d22-e1b1-4d81-9200-8d858a45a51e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.072842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] The token cache is being invalidate because project 03c81d7d451f441d8fd192c8c3b7ab8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:12.073208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.073936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.075653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 912/1788] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:11 2022] DELETE /identity/v3/projects/03c81d7d451f441d8fd192c8c3b7ab8c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:12.080442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:12.081854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:12.106953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 877/1789] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:11 2022] DELETE /identity/v3/users/0da9385ae3ff4d3f9674b24de8f0d341 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:12.112099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:12.112534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:12.112908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:12.132669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:12.133936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:12.155507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774d22-e1b1-4d81-9200-8d858a45a51e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:12.208157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 878/1790] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:12.214655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:12.215546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:12.215886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:12.216168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] PATH_INFO: `/v3/users/c1373425b33d448ba86661d6f21f4335` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:12.217706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1373425b33d448ba86661d6f21f4335)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:12.218699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:12.220253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:12.220611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:12.284413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593a3b4-3bde-4e3d-9e07-39ee65dd24f9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:12.288608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:12.289658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:12.290101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:12.290417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:12.301450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:12.308805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:12.346925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 879/1791] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:12.396108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1373425b33d448ba86661d6f21f4335'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.397386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1373425b33d448ba86661d6f21f4335'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.417073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1373425b33d448ba86661d6f21f4335'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.424374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1373425b33d448ba86661d6f21f4335'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.425314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1373425b33d448ba86661d6f21f4335'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:12.495152 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 913/1792] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:12 2022] DELETE /identity/v3/users/c1373425b33d448ba86661d6f21f4335 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:13.740551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:13.744211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:13.788113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05af3a-1c65-4f75-809b-8c37099d8794 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:13.816834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:13.819131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:14.062948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:14.066040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.068646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.068646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] PATH_INFO: `/v3/projects/e1989d8c870646fe94d2358e0cedf151` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.073152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1989d8c870646fe94d2358e0cedf151)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:14.076819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:14.116865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91249d-ad87-421a-9cd2-23d9489eedc7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:14.124563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:14.127397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.128971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.129772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] PATH_INFO: `/v3/projects/3bf611fe134c45d18022145f1ed541d7` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.132972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf611fe134c45d18022145f1ed541d7)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:14.135639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:14.223473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1989d8c870646fe94d2358e0cedf151', 'request_id': 'req-317b00a7-9255-4087-ae04-68a3ff085b2e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.315106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf611fe134c45d18022145f1ed541d7', 'request_id': 'req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.378709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1989d8c870646fe94d2358e0cedf151', 'request_id': 'req-317b00a7-9255-4087-ae04-68a3ff085b2e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.390612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1989d8c870646fe94d2358e0cedf151', 'request_id': 'req-317b00a7-9255-4087-ae04-68a3ff085b2e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.429605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] The token cache is being invalidate because project e1989d8c870646fe94d2358e0cedf151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:14.429926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.430911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-317b00a7-9255-4087-ae04-68a3ff085b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.433081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 880/1793] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:13 2022] DELETE /identity/v3/projects/e1989d8c870646fe94d2358e0cedf151 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:14.434464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.434751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.435029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.439524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 881/1794] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:17:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:17:14.440565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:14.443199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:14.489067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf611fe134c45d18022145f1ed541d7', 'request_id': 'req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.498368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf611fe134c45d18022145f1ed541d7', 'request_id': 'req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.525626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] The token cache is being invalidate because project 3bf611fe134c45d18022145f1ed541d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:14.526028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.527718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-dd5e2fb5-15e6-4701-bafb-5c310904d0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.532941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 914/1795] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:13 2022] DELETE /identity/v3/projects/3bf611fe134c45d18022145f1ed541d7 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:14.534509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0608ba-44f5-4be8-96a3-3ef27da24fc0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.534927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0608ba-44f5-4be8-96a3-3ef27da24fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.535310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0608ba-44f5-4be8-96a3-3ef27da24fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.575285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4e0608ba-44f5-4be8-96a3-3ef27da24fc0 None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:14.576667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4e0608ba-44f5-4be8-96a3-3ef27da24fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:14.580609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719541-20bc-4cce-ab34-a0f5a3ecd34b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:14.599232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:14.601536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.602060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.602471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] PATH_INFO: `/v3/users/0dd49a8605cd44d79a53ce79e4cbb28d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.604071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd49a8605cd44d79a53ce79e4cbb28d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:14.606695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:14.677793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 915/1796] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:14.679255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188d2d1-8be9-4992-8498-cf38ef5d872d None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.679551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188d2d1-8be9-4992-8498-cf38ef5d872d None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.680147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188d2d1-8be9-4992-8498-cf38ef5d872d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.711684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3188d2d1-8be9-4992-8498-cf38ef5d872d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:14.713458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3188d2d1-8be9-4992-8498-cf38ef5d872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:14.737830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 916/1797] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:14.745386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.746114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.746114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.784861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:14.786113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:14.799717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 917/1798] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:14.801146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:14.816951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a982-6157-4d39-804c-44ab4e059a9b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:14.836957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:14.841829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.842720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.842720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.843237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:14.843727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:14.845507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:14.882916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 918/1799] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:14.886859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:14.899897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd49a8605cd44d79a53ce79e4cbb28d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.901171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd49a8605cd44d79a53ce79e4cbb28d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.911274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56367bb0-6376-4dc1-924b-139bdcb99180 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:14.928272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd49a8605cd44d79a53ce79e4cbb28d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.930007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd49a8605cd44d79a53ce79e4cbb28d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:14.934834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:14.938137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:14.939136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:14.939585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:14.940638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:14.945574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:14.949404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd49a8605cd44d79a53ce79e4cbb28d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:15.057265 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 882/1800] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:14 2022] DELETE /identity/v3/users/0dd49a8605cd44d79a53ce79e4cbb28d => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:15.088274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 919/1801] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:14 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:15.092769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.094784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:15.190432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a90750-e114-4d5f-b82c-942e49e62159 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.256852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.258552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:15.299561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.300384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.300822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.301204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.303429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.308780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.360151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd3d7-99ed-4e67-a96d-3f482834f1ad admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.366185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.366933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.367263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.367486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.369325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:15.382979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.384761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.396055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 920/1802] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:15.414616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 883/1803] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:15 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:15.417521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.437074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e92ac6-b4a1-4f69-a1f5-387e59ede776 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.449695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.450166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.450736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.451023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.451768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.457294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.487796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 921/1804] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:15.492101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.506966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a3a8-b6eb-4cff-8a66-1e4770041a05 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.514878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.515768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.516195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.516685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.518488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.520254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb78b48-fe69-4a83-b5ed-70a059fe478c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.535305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 884/1805] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:15.539281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.553293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b8fe-9f9e-412c-8f72-137fed9f50a5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.557797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.558819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.559197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.559517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] PATH_INFO: `/v3/projects/e1cbadd25f7d43cea796f9f79b30ed78/users/3f818aa14e9647639d05bd528bdf45de/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.576454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cbadd25f7d43cea796f9f79b30ed78, user_id=3f818aa14e9647639d05bd528bdf45de, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.580573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.663891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 922/1806] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:15 2022] PUT /identity/v3/projects/e1cbadd25f7d43cea796f9f79b30ed78/users/3f818aa14e9647639d05bd528bdf45de/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:15.672779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.674078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.674078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.700461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] MFA Rules not processed for user `3f818aa14e9647639d05bd528bdf45de`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:15.701750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:15.730986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.733766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:15.789834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 885/1807] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:15.794702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:15.809525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4130-f6c3-4ea5-ab7a-032ceb353eb6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.816192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.817228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.818032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.818189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] PATH_INFO: `/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.820008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5bbcdeb7c8947ed8d37beb43205dc14)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.823067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.867898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820f0ee-1a15-4325-bba2-f702dd0d2270 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:15.872433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:15.873091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:15.873368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:15.873639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:15.874374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:15.876400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:15.963574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5bbcdeb7c8947ed8d37beb43205dc14', 'request_id': 'req-918f59ec-8926-4c74-bf42-508bdcee4a4c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:15.964155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 886/1808] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:15 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:15.967883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.025743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b50da-86b9-4d16-ab31-2e96274f2276 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.025941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5bbcdeb7c8947ed8d37beb43205dc14', 'request_id': 'req-918f59ec-8926-4c74-bf42-508bdcee4a4c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:16.030340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.031271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.031549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.031827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.032284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5bbcdeb7c8947ed8d37beb43205dc14', 'request_id': 'req-918f59ec-8926-4c74-bf42-508bdcee4a4c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:16.032502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.034433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.044001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] The token cache is being invalidate because project f5bbcdeb7c8947ed8d37beb43205dc14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:16.044296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:16.045005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-918f59ec-8926-4c74-bf42-508bdcee4a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:16.046742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 923/1809] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:15 2022] DELETE /identity/v3/projects/f5bbcdeb7c8947ed8d37beb43205dc14 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:16.051655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f260f9-f3c2-4cee-a4e5-f38b57842804 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.051948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f260f9-f3c2-4cee-a4e5-f38b57842804 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.052235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f260f9-f3c2-4cee-a4e5-f38b57842804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.070683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a8f260f9-f3c2-4cee-a4e5-f38b57842804 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:16.071888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a8f260f9-f3c2-4cee-a4e5-f38b57842804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:16.080443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 924/1810] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:16.088066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.088359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.088640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.106032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 887/1811] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:15 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:16.107746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:16.109032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:16.110051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.113105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:16.124599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 925/1812] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:16.129087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035948c5-e64b-438c-a7f6-24ad8ef1b242 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.132348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.133982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.134707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.134991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.135265 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.135991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.137758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.146898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8e00-e482-420e-a638-5c88542c0a29 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.152595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.153662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.154188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 888/1813] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:16.154319 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.154907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.156012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:16.156392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.158724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.158962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.170239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dcc61-b19b-4c8d-bd83-db086191056b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.177628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 926/1814] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:16.179500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.179734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.180684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.181027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.181360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] PATH_INFO: `/v3/projects/be063f3a2e864c33a0ecdd41542067c9/users/2122e1a600c04a2fa7de87616ffcb0c4/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.192244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21aa9-c3ae-476c-82bd-d53ad46d7690 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.197814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.198520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.198811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.199202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.200023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.201976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.202556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be063f3a2e864c33a0ecdd41542067c9, user_id=2122e1a600c04a2fa7de87616ffcb0c4, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.207290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.335225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 889/1815] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:16 2022] PUT /identity/v3/projects/be063f3a2e864c33a0ecdd41542067c9/users/2122e1a600c04a2fa7de87616ffcb0c4/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:16.339632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.369920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 927/1816] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:16.371160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.421109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00ae6f-7a0f-4952-a632-859832a3c2b4 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.448809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aea15b-e561-46f5-8437-3f9ec0183942 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.454026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.455346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.455627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.455951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.456795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.458950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.511276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.512521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.512870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.513215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.514013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.515744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.527371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 890/1817] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:16.531656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.542816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce84ec1-4591-4606-af09-7e62bac08710 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.549390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.550431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.551046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.551587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] PATH_INFO: `/v3/projects/be063f3a2e864c33a0ecdd41542067c9/users/2122e1a600c04a2fa7de87616ffcb0c4/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.553886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be063f3a2e864c33a0ecdd41542067c9, user_id=2122e1a600c04a2fa7de87616ffcb0c4, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.556642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44a6ce-b0d4-4462-9773-0623c583a90f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.569326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 891/1818] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:16 2022] PUT /identity/v3/projects/be063f3a2e864c33a0ecdd41542067c9/users/2122e1a600c04a2fa7de87616ffcb0c4/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:16.580091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.580539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.580800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.593621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 928/1819] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/users => generated 305 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:16.601809 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.614220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] MFA Rules not processed for user `2122e1a600c04a2fa7de87616ffcb0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:16.615772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:16.617972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07d03b-2c01-4c8d-8ca6-54ea79e8b0d6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.696672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.697831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 892/1820] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:16.698458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.698974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.699177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.700227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.703641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.716184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 929/1821] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:16.718629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.719020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.725683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:16.744571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ea649-3683-44a8-9d4b-7baebe15ef75 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.753191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.754171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.754636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.755736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.756746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.758753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.773086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 930/1822] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:16.778877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.793819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e18a50-053a-48e4-bbc0-0f72f4c305f3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.800486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.802591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.803352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.803851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] PATH_INFO: `/v3/projects/ee286cfb02ba4b8f9dd748550965a57b/users/9b12dd6d17ba47c09106768a91551d37/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.809237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484330-41c0-42ef-9ade-acd6529ef918 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.813231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.814926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.815228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.815546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.824556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee286cfb02ba4b8f9dd748550965a57b, user_id=9b12dd6d17ba47c09106768a91551d37, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.825590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.827136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.828544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e902e64-694f-4087-a5e3-6d7878ad6a9c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.837903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 893/1823] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:16.870345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:16.892926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1540-f542-4b24-bc8a-858113b3aa15 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:16.906188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:16.908724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.910018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.912462 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] PATH_INFO: `/v3/users/3f818aa14e9647639d05bd528bdf45de` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.916699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f818aa14e9647639d05bd528bdf45de)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:16.918759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 931/1824] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:16 2022] PUT /identity/v3/projects/ee286cfb02ba4b8f9dd748550965a57b/users/9b12dd6d17ba47c09106768a91551d37/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:16.928491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:16.933017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:16.933347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:16.933631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:16.987936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] MFA Rules not processed for user `9b12dd6d17ba47c09106768a91551d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:16.989373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:17.090692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 932/1825] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:17.097582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.185166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71418c39-be0b-4c80-8719-94737cf3d9c9 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.192723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.192723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.195475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.195475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.195475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.196970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.198868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f818aa14e9647639d05bd528bdf45de'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:17.200147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f818aa14e9647639d05bd528bdf45de'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:17.222347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f818aa14e9647639d05bd528bdf45de'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:17.223708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f818aa14e9647639d05bd528bdf45de'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:17.238824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-1e266d6e-c978-4ef0-8804-27078296146a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f818aa14e9647639d05bd528bdf45de'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:17.322095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 894/1826] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:16 2022] DELETE /identity/v3/users/3f818aa14e9647639d05bd528bdf45de => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:17.326620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 933/1827] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:17 2022] POST /identity/v3/projects => generated 409 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:17.326766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.326941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.327220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.331085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.332929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:17.370466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:17.372332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:17.464833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 895/1828] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:17.492877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34045-138c-4a91-993a-af601e21925d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.497798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.498520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.498793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.500645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.501564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.503701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.507258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.509186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:17.583544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72548c5-43d9-43fc-931a-51cebc833d2f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.587987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.588650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.588983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.589263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.601351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.603178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.613559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 934/1829] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:17 2022] POST /identity/v3/users => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:17.614913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.618120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 896/1830] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:17.628093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd55a5-b507-414d-88e5-6d49a1bb43fd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.634562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.635495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.636722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.636722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.638233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.640396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.654984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 935/1831] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:17.660049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.670693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa8a9d-f4b1-40d9-abc4-9717d7cd486d service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.675674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.676567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.676850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.677158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] PATH_INFO: `/v3/projects/7261f2bda18d40c68261acfda6c136ca/users/0bb94d5ce6c6477ab35934820950093d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.695210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7261f2bda18d40c68261acfda6c136ca, user_id=0bb94d5ce6c6477ab35934820950093d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.698048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.778568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 897/1832] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:17 2022] PUT /identity/v3/projects/7261f2bda18d40c68261acfda6c136ca/users/0bb94d5ce6c6477ab35934820950093d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:17.783262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.797688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140745a-560c-495a-87d5-89d52db6dae3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.891270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.892744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.893179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.893460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.894217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.895842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1743c7-5f2e-4421-ba4b-708b5638b7c0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.906348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 936/1833] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:17.911223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:17.924561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9efd7-abeb-4f5c-a773-35886e0ed4db admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:17.930001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:17.930982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.931306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.931596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] PATH_INFO: `/v3/projects/7261f2bda18d40c68261acfda6c136ca/users/0bb94d5ce6c6477ab35934820950093d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:17.933902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7261f2bda18d40c68261acfda6c136ca, user_id=0bb94d5ce6c6477ab35934820950093d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:17.936638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:17.962887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 898/1834] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:17 2022] PUT /identity/v3/projects/7261f2bda18d40c68261acfda6c136ca/users/0bb94d5ce6c6477ab35934820950093d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:17.967107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:17.967107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:17.967107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:18.138651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] MFA Rules not processed for user `0bb94d5ce6c6477ab35934820950093d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:18.145000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:18.224935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 937/1835] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:18.237659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:18.239511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:18.317457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f394cb-5b92-4aee-ba1b-a78b69099e62 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:18.322830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:18.323656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:18.324042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:18.324542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:18.339355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:18.340993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:18.354132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 899/1836] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:18.468716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:18.470302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:18.538992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbf28c-b8fd-4dde-9096-b3c55029aa1a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:18.544365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:18.545269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:18.545595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:18.545970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] PATH_INFO: `/v3/projects/e1cbadd25f7d43cea796f9f79b30ed78` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:18.547160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1cbadd25f7d43cea796f9f79b30ed78)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:18.552370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:18.678230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1cbadd25f7d43cea796f9f79b30ed78', 'request_id': 'req-34944652-06f2-4303-9abf-8c37980ad2ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:18.752508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1cbadd25f7d43cea796f9f79b30ed78', 'request_id': 'req-34944652-06f2-4303-9abf-8c37980ad2ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:18.759868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1cbadd25f7d43cea796f9f79b30ed78', 'request_id': 'req-34944652-06f2-4303-9abf-8c37980ad2ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:18.777010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] The token cache is being invalidate because project e1cbadd25f7d43cea796f9f79b30ed78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:18.779839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:18.781540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:18.784098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 938/1837] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:18 2022] DELETE /identity/v3/projects/e1cbadd25f7d43cea796f9f79b30ed78 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:18.787955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:18.801599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be1a1-6978-4085-9804-e7aa4a7d8913 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:18.814477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:18.815970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:18.909083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34944652-06f2-4303-9abf-8c37980ad2ea admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:18.913567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:18.914282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:18.914643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:18.914927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:18.919301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:18.944162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:18.944976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:18.945397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:18.945681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] PATH_INFO: `/v3/users/2122e1a600c04a2fa7de87616ffcb0c4` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:18.948827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2122e1a600c04a2fa7de87616ffcb0c4)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:18.950788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:19.008081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:19.009681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:19.031479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 939/1838] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:19.108580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2122e1a600c04a2fa7de87616ffcb0c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:19.109236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2122e1a600c04a2fa7de87616ffcb0c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:19.129649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2122e1a600c04a2fa7de87616ffcb0c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:19.130427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2122e1a600c04a2fa7de87616ffcb0c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:19.137538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d4660c36-d974-4b45-9d3f-207d052e86bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2122e1a600c04a2fa7de87616ffcb0c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:19.192808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 900/1839] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:18 2022] DELETE /identity/v3/users/2122e1a600c04a2fa7de87616ffcb0c4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:19.781769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:19.783865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:19.797311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7f8d-de45-4418-b992-ed83ae69592d service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:19.855828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:19.856689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:19.856965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:19.857237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] PATH_INFO: `/v3/projects/be063f3a2e864c33a0ecdd41542067c9` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:19.858440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=be063f3a2e864c33a0ecdd41542067c9)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:19.861372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:19.967996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be063f3a2e864c33a0ecdd41542067c9', 'request_id': 'req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:20.048559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be063f3a2e864c33a0ecdd41542067c9', 'request_id': 'req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:20.054642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be063f3a2e864c33a0ecdd41542067c9', 'request_id': 'req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:20.073275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] The token cache is being invalidate because project be063f3a2e864c33a0ecdd41542067c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:20.073566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:20.074248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5c1bfb3d-fb6a-47e1-ae7b-7ed4cbade9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:20.075897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 940/1840] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:19 2022] DELETE /identity/v3/projects/be063f3a2e864c33a0ecdd41542067c9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:20.080804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.081300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.081389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.100074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:20.101285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:20.165422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 901/1841] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:20.173711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.173996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.174442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.192934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:20.194135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:20.202263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 941/1842] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:20.207239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.217218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788a0b-5891-42a6-959c-5a20a0530bf5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.222672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.223278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.223887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.224174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.224976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:20.225287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.226912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.242624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 902/1843] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:20.244044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.254411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a0236-aab5-4c2a-9a3d-1ab0caad9fb3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.268635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.269414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.269795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.270158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.271032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.272980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.375880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 942/1844] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:20 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:20.380601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.390434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517f917-cb89-4ce2-b182-b649e94f7492 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.446662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.447341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.447649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.448067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.448758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.450717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.540630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 903/1845] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:20 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:20.544716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.560027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78dee8-d30d-49af-b5f2-33a616adc70f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.565849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.567127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.567127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.567127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.567658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.569288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.581362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 943/1846] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:20.587475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.598492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3b0d6-89b5-41ca-94ca-5b8ede6d0f15 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.609267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.610049 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.613104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.613104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.614213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.616129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8b94a-356d-4a33-b9de-84679f332f1e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.637424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 904/1847] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:20.642641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:20.653427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b8215-ccfd-4710-b786-758240568ccc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:20.672931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:20.675982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.676279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.676561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] PATH_INFO: `/v3/projects/cfe7753fa1704102b289a1ad4fa86d3b/users/d62bc7e7a64946c69e4d930dac21b0cb/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.695012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe7753fa1704102b289a1ad4fa86d3b, user_id=d62bc7e7a64946c69e4d930dac21b0cb, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:20.697653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:20.907922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 944/1848] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:20 2022] PUT /identity/v3/projects/cfe7753fa1704102b289a1ad4fa86d3b/users/d62bc7e7a64946c69e4d930dac21b0cb/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:20.921406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:20.921702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:20.921984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:20.948374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] MFA Rules not processed for user `d62bc7e7a64946c69e4d930dac21b0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:20.949709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:21.015739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 905/1849] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:21.021081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.083005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aca43-ecfd-4721-a051-e7018fd72a43 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.087633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.088287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.088573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.088859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.089541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.091455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.186756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 945/1850] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:21 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:21.191662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.202231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5dfc9-dd30-46c0-b593-f4a4b1372ae3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.256896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.257573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.257855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.258133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.258856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.260772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.356089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 906/1851] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:21 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:21.357216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.368293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b7cb-6a94-42eb-a59d-cc63cd5b8b3d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.372932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.373597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.373885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.374162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.374918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.377155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.387663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 946/1852] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:21.393214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.403285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a96db1-7cf7-4cfb-bf8b-478d47a71efb admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.408315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.409204 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.409493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.409772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] PATH_INFO: `/v3/projects/3bc752df372147638d01f159141a38f9/users/ab7f915a06d342748c94dc81c21196a3/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.424644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc752df372147638d01f159141a38f9, user_id=ab7f915a06d342748c94dc81c21196a3, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.427408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.490116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 907/1853] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:21 2022] PUT /identity/v3/projects/3bc752df372147638d01f159141a38f9/users/ab7f915a06d342748c94dc81c21196a3/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:21.494605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.505817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc78e-8894-469c-a5e5-9faf0365e0f9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.559786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.561088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.561379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.561695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.562462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.564796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586b29-84f7-4ad0-8fd6-c18c354d73e9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.574502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 947/1854] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:21.578722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.592669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e4f1d-abf3-4a97-89eb-97a1f86ac731 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.599657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.600591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.600875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.601151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] PATH_INFO: `/v3/projects/3bc752df372147638d01f159141a38f9/users/ab7f915a06d342748c94dc81c21196a3/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.603172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc752df372147638d01f159141a38f9, user_id=ab7f915a06d342748c94dc81c21196a3, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.605873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.617234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 908/1855] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:21 2022] PUT /identity/v3/projects/3bc752df372147638d01f159141a38f9/users/ab7f915a06d342748c94dc81c21196a3/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:21.628903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.629189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.630918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.656644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] MFA Rules not processed for user `ab7f915a06d342748c94dc81c21196a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:21.658219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:21.726489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 948/1856] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:21.737314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:21.743098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:21.804861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5008e-e1fc-4484-9eac-b9f3fe6d94a2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:21.804861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:21.804861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:21.804861 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:21.805514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:21.820173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:21.821624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6f4f2-36c3-4742-bbd8-9f00f62beb25 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:21.842395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 909/1857] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:23.050698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:23.122003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54a33b-9d9d-4583-b937-4bbcf0acb466 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:23.127154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:23.128018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:23.128293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:23.128564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] PATH_INFO: `/v3/users/d62bc7e7a64946c69e4d930dac21b0cb` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:23.129762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62bc7e7a64946c69e4d930dac21b0cb)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:23.131707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:23.291745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62bc7e7a64946c69e4d930dac21b0cb'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:23.292839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62bc7e7a64946c69e4d930dac21b0cb'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:23.307069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62bc7e7a64946c69e4d930dac21b0cb'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:23.315419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62bc7e7a64946c69e4d930dac21b0cb'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:23.316218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62bc7e7a64946c69e4d930dac21b0cb'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:23.380863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 949/1858] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:23 2022] DELETE /identity/v3/users/d62bc7e7a64946c69e4d930dac21b0cb => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:23.385367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:23.385829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:23.385917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:23.403430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:23.404836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:23.475200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 910/1859] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:23.484975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:23.486999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:23.549068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800790c4-15c2-4859-bb10-871810320532 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:23.552953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:23.553613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:23.553902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:23.554181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:23.564449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:23.566014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:23.576245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 950/1860] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:24.136299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:24.137714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:24.150448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04a864-ef3d-469c-9077-fe69c4914ef1 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:24.155542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:24.156679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:24.156963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:24.157672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] PATH_INFO: `/v3/projects/cfe7753fa1704102b289a1ad4fa86d3b` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:24.158905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe7753fa1704102b289a1ad4fa86d3b)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:24.160852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:24.277721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe7753fa1704102b289a1ad4fa86d3b', 'request_id': 'req-4cab7165-dc87-4d6b-814b-304c4df1b93d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.322826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe7753fa1704102b289a1ad4fa86d3b', 'request_id': 'req-4cab7165-dc87-4d6b-814b-304c4df1b93d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.328570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe7753fa1704102b289a1ad4fa86d3b', 'request_id': 'req-4cab7165-dc87-4d6b-814b-304c4df1b93d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.339867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] The token cache is being invalidate because project cfe7753fa1704102b289a1ad4fa86d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:24.340146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.340870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.342444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 911/1861] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:24 2022] DELETE /identity/v3/projects/cfe7753fa1704102b289a1ad4fa86d3b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:24.346848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:24.356230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb468aa-4ee5-4848-b136-0cd702d3cad6 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:24.411756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:24.412560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:24.412816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:24.413084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] PATH_INFO: `/v3/users/ab7f915a06d342748c94dc81c21196a3` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:24.414277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7f915a06d342748c94dc81c21196a3)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:24.416175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:24.640004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7f915a06d342748c94dc81c21196a3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.640681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7f915a06d342748c94dc81c21196a3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.653679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7f915a06d342748c94dc81c21196a3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.662172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7f915a06d342748c94dc81c21196a3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.662856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1632da9-5df9-4729-a40f-b30fd308d3d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7f915a06d342748c94dc81c21196a3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:24.760309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 951/1862] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:24 2022] DELETE /identity/v3/users/ab7f915a06d342748c94dc81c21196a3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:25.153422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.154895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:25.171441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7165-dc87-4d6b-814b-304c4df1b93d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.240552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.241436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.241720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.242001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] PATH_INFO: `/v3/projects/3bc752df372147638d01f159141a38f9` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.243255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc752df372147638d01f159141a38f9)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.245317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:25.357651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc752df372147638d01f159141a38f9', 'request_id': 'req-16faba89-33dd-44ac-91eb-7286f2f8f240'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:25.437201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc752df372147638d01f159141a38f9', 'request_id': 'req-16faba89-33dd-44ac-91eb-7286f2f8f240'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:25.443974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc752df372147638d01f159141a38f9', 'request_id': 'req-16faba89-33dd-44ac-91eb-7286f2f8f240'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:25.455561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] The token cache is being invalidate because project 3bc752df372147638d01f159141a38f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:25.455788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:25.456456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16faba89-33dd-44ac-91eb-7286f2f8f240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:25.458062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 912/1863] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:25 2022] DELETE /identity/v3/projects/3bc752df372147638d01f159141a38f9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:25.462947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.463430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.463520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.481939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:25.483153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:25.540368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 952/1864] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:25.548147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.548432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.548709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.568058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:25.569314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:25.577341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 913/1865] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:25.582428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.592155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30210b20-808b-4fb9-a59e-33c2fa7cb5b3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.597296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.597943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.598254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.598533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.599256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:25.599562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.601188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:25.613115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 953/1866] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:25.617431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.627142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bafd35-870a-4f0e-a363-3f3ca0924bfe None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.631793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.632443 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.632728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.633005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.633690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.635627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:25.771686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 914/1867] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:25 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:25.776047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.786949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735d66-7f0a-4edd-917a-8eb0edda478d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.843581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.844271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.844545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.844842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.845823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.847858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:25.935241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 954/1868] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:25 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:25.940256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.951230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e9d98-a4fe-4c62-a832-5db43265aa08 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.956573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.957199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.957460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.957734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.958551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.960133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:25.971231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 915/1869] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:25.975253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:25.985120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de11a-803e-419a-b4e5-a420d550626a admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:25.989894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:25.990594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:25.990877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:25.991151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:25.991879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:25.993462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f399ae-2679-4be2-be27-5b7f740632ab admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.003975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 955/1870] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:26.007967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.018348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a61f13-1d77-471b-839b-93c50e5833a5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.023490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.024382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.024667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.024945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] PATH_INFO: `/v3/projects/728d6be0ab0e4e0580a4cd10fd71f6e4/users/c3847a8c9e08401dafeb3b180b3974a9/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.039963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728d6be0ab0e4e0580a4cd10fd71f6e4, user_id=c3847a8c9e08401dafeb3b180b3974a9, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.042642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.146573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 916/1871] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:26 2022] PUT /identity/v3/projects/728d6be0ab0e4e0580a4cd10fd71f6e4/users/c3847a8c9e08401dafeb3b180b3974a9/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:26.156901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880209c-cc47-4530-a95b-708327b90dab None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.157193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880209c-cc47-4530-a95b-708327b90dab None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.157615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880209c-cc47-4530-a95b-708327b90dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.184519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2880209c-cc47-4530-a95b-708327b90dab None None] MFA Rules not processed for user `c3847a8c9e08401dafeb3b180b3974a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:26.185664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2880209c-cc47-4530-a95b-708327b90dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:26.248815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 956/1872] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:26.253380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.316631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ebd3-277a-4139-81f9-735cad7f5cb2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.321590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.322294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.322580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.322857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.323555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.325456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.422155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 917/1873] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:26 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:26.427039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880209c-cc47-4530-a95b-708327b90dab None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.439136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880209c-cc47-4530-a95b-708327b90dab None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.499192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.499868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.500376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.500721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.502020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.504089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.580975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 957/1874] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:26 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:26.585501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.598655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4510efa-f104-4a65-b071-c54c5cd8d983 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.604067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.605085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.605373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.605657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.606620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.608181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.621171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 918/1875] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:26.625381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.636027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f33178-39fb-4135-a928-9d17910a3aae admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.640860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.641820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.643020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.643120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] PATH_INFO: `/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34/users/75a5953545e4498c9f67ffea7e408cdb/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.657803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d05ee9a3af14d0a8aa1b67a26fd5e34, user_id=75a5953545e4498c9f67ffea7e408cdb, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.660573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.747477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 958/1876] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:26 2022] PUT /identity/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34/users/75a5953545e4498c9f67ffea7e408cdb/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:26.752933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.763046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49bd7-d7e4-44c1-9cc2-4b4c0f0f248e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.825188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.825937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.826258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.826544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.827293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.828893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb54d9-b278-474b-a301-735573d9e0c7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.840030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 919/1877] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:26.844187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:26.855026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dae0-27a2-429c-afd3-68a24e377a6d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:26.859722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:26.861156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.861430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.861699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] PATH_INFO: `/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34/users/75a5953545e4498c9f67ffea7e408cdb/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.863735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d05ee9a3af14d0a8aa1b67a26fd5e34, user_id=75a5953545e4498c9f67ffea7e408cdb, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:26.866752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:26.877745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 959/1878] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:26 2022] PUT /identity/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34/users/75a5953545e4498c9f67ffea7e408cdb/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:26.894135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:26.894472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:26.894751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:26.925257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] MFA Rules not processed for user `75a5953545e4498c9f67ffea7e408cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:26.926931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:27.008390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 920/1879] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:27.015722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:27.019781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:27.081605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae8966-28cc-4ee8-8b54-82924f6f35f7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:27.085565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:27.086353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:27.086707 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:27.087053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:27.097080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:27.098660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2254b-b4a8-421e-8d83-244f25c9ec1b service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:27.109038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 960/1880] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:27.736479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:27.870868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd83ad-15e4-4e51-9865-2a07e1b1d545 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:27.875735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:27.876933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:27.877211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:27.877486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] PATH_INFO: `/v3/users/c3847a8c9e08401dafeb3b180b3974a9` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:27.878723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3847a8c9e08401dafeb3b180b3974a9)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:27.880711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:28.113321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3847a8c9e08401dafeb3b180b3974a9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:28.114020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3847a8c9e08401dafeb3b180b3974a9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:28.126926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3847a8c9e08401dafeb3b180b3974a9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:28.127729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3847a8c9e08401dafeb3b180b3974a9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:28.134616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3847a8c9e08401dafeb3b180b3974a9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:28.261200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 921/1881] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:27 2022] DELETE /identity/v3/users/c3847a8c9e08401dafeb3b180b3974a9 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:28.265745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:28.266837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:28.267356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:28.285360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:28.286622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:28.373422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 961/1882] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:28.417591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:28.419398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:28.491185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042958c-2154-4a56-9611-8cd66922fff6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:28.496571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:28.497456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:28.497746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:28.498030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:28.512450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:28.514821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:28.530570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 922/1883] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:29.627832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:29.629273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:29.660551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacd23-f72e-47d0-8061-6e538b09694e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:29.678170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:29.681828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:29.681828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:29.681828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] PATH_INFO: `/v3/projects/728d6be0ab0e4e0580a4cd10fd71f6e4` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:29.682156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728d6be0ab0e4e0580a4cd10fd71f6e4)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:29.685676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:29.810247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728d6be0ab0e4e0580a4cd10fd71f6e4', 'request_id': 'req-51acf152-7f86-4bcc-92c8-8ea980b47dd5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:29.894872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728d6be0ab0e4e0580a4cd10fd71f6e4', 'request_id': 'req-51acf152-7f86-4bcc-92c8-8ea980b47dd5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:29.901362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728d6be0ab0e4e0580a4cd10fd71f6e4', 'request_id': 'req-51acf152-7f86-4bcc-92c8-8ea980b47dd5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:29.916874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] The token cache is being invalidate because project 728d6be0ab0e4e0580a4cd10fd71f6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:29.917721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:29.918896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:29.921542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 962/1884] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:29 2022] DELETE /identity/v3/projects/728d6be0ab0e4e0580a4cd10fd71f6e4 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:29.926147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:29.936675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbe0d-2a06-4ba1-bfca-4f3baf9084db service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:30.004577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:30.005490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:30.005807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:30.006085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] PATH_INFO: `/v3/users/75a5953545e4498c9f67ffea7e408cdb` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:30.007382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a5953545e4498c9f67ffea7e408cdb)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:30.009336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:30.206184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a5953545e4498c9f67ffea7e408cdb'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.206936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a5953545e4498c9f67ffea7e408cdb'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.218959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a5953545e4498c9f67ffea7e408cdb'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.219661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a5953545e4498c9f67ffea7e408cdb'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.226935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e338cb55-cf71-473c-b57f-34a9999f2a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a5953545e4498c9f67ffea7e408cdb'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.289787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 923/1885] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:29 2022] DELETE /identity/v3/users/75a5953545e4498c9f67ffea7e408cdb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:30.730847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:30.732357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:30.747397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acf152-7f86-4bcc-92c8-8ea980b47dd5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:30.805923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:30.806818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:30.807098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:30.807373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] PATH_INFO: `/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:30.808545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d05ee9a3af14d0a8aa1b67a26fd5e34)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:30.810523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:30.911052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d05ee9a3af14d0a8aa1b67a26fd5e34', 'request_id': 'req-035943f2-d3e7-4c5b-83db-578f20a5a2c2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.969305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d05ee9a3af14d0a8aa1b67a26fd5e34', 'request_id': 'req-035943f2-d3e7-4c5b-83db-578f20a5a2c2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.976260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d05ee9a3af14d0a8aa1b67a26fd5e34', 'request_id': 'req-035943f2-d3e7-4c5b-83db-578f20a5a2c2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.989357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] The token cache is being invalidate because project 6d05ee9a3af14d0a8aa1b67a26fd5e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:30.989652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.990361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-035943f2-d3e7-4c5b-83db-578f20a5a2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:30.991928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 963/1886] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:30 2022] DELETE /identity/v3/projects/6d05ee9a3af14d0a8aa1b67a26fd5e34 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:30.997405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:30.997910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:30.998099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.018157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:31.019526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.086248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 924/1887] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:31.089975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.090348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.090637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.109413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:31.110841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.119379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 964/1888] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:31.123524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.133951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405efddb-4f53-4201-8a7f-c1094321941e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.138680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.139346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.139629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.139906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.140634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:31.140943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.143326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.155553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 925/1889] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:31.161739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.171016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef5651-faa7-491d-9468-efdca81a87d4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.176001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.176658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.176987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.177246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.177999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.179965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.275506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 965/1890] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:31 2022] POST /identity/v3/projects => generated 361 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:31.280029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.291821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87cb29-0c14-4690-a6d0-e102a7414aed admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.357538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.358589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.358848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.359125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.359814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.363165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.381216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.383537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.398024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29878fd9-0bd2-48d8-b22a-abd493b3cd2e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.404184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.405896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.405896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.405896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] PATH_INFO: `/v3/users/9b12dd6d17ba47c09106768a91551d37` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.407513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b12dd6d17ba47c09106768a91551d37)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.410468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.462112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 926/1891] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:31 2022] POST /identity/v3/users => generated 281 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:31.466562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.480249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206e2dc-57d7-4162-88f3-bab1c8f27642 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.487266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.488017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.488307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.488583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.489321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.491505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.502686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 927/1892] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:31.507381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.518126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dc329-4b2d-447c-acea-53b836fb346d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.523450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.524280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.524739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.525077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.525918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.527742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.547305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 928/1893] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:31.549674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.560891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce58b1-5cc0-427b-b090-5ffe59364cb5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.568114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.569055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.572321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.572321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] PATH_INFO: `/v3/projects/64173a599d434a8c95a91276baac03aa/users/a4b33f18d14a49d28bc35f5558ae165d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.586098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64173a599d434a8c95a91276baac03aa, user_id=a4b33f18d14a49d28bc35f5558ae165d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.588810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d93b92-c9e1-49cd-bee5-36e4cd7713a0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.615928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b12dd6d17ba47c09106768a91551d37'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:31.616637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b12dd6d17ba47c09106768a91551d37'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:31.629084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b12dd6d17ba47c09106768a91551d37'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:31.635956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b12dd6d17ba47c09106768a91551d37'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:31.636659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de075f21-c363-498c-a1e2-1a0c556485f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b12dd6d17ba47c09106768a91551d37'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:31.682875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 929/1894] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:31 2022] PUT /identity/v3/projects/64173a599d434a8c95a91276baac03aa/users/a4b33f18d14a49d28bc35f5558ae165d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:31.694830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.695310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.695791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.730137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 966/1895] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:31 2022] DELETE /identity/v3/users/9b12dd6d17ba47c09106768a91551d37 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:31.732368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] MFA Rules not processed for user `a4b33f18d14a49d28bc35f5558ae165d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:31.733748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.735099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.735635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.736509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.754464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:31.756141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.801351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 930/1896] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:31.827853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 967/1897] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:31.837988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.839418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.839984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.841324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.900564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703ecf6-359d-4918-a842-1b2c2ee5a6ff None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.904519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.905168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.905444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.905713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.905806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb95c3-4e91-4986-859c-61f611001c1b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.910756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.911399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.911678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.911951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.915779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.917209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.920103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.921500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02066b5-3f37-45ee-b70b-11bc97f12bad service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:31.927109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 968/1898] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:31.932053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 931/1899] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:31.946736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:31.948197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:31.960692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19067183-d256-4ca9-9eac-79ffd10a11fd service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:31.965768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:31.966753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:31.967100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:31.967440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] PATH_INFO: `/v3/users/a4b33f18d14a49d28bc35f5558ae165d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:31.968695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b33f18d14a49d28bc35f5558ae165d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:31.971226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:32.160705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b33f18d14a49d28bc35f5558ae165d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.161375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b33f18d14a49d28bc35f5558ae165d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.173798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b33f18d14a49d28bc35f5558ae165d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.181253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b33f18d14a49d28bc35f5558ae165d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.181982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b33f18d14a49d28bc35f5558ae165d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.265454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 969/1900] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:31 2022] DELETE /identity/v3/users/a4b33f18d14a49d28bc35f5558ae165d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:32.270471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:32.271866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:32.272149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:32.290994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:32.292872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:32.353396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 932/1901] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:32.365643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:32.367478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:32.434289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0efdd3-a043-4d0f-8a8e-334839dacc87 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:32.437997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:32.439268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:32.439588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:32.440153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:32.449825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:32.451324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:32.463978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:32.465480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:32.466191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 970/1902] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:32.480408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222e697-3f94-49b5-b884-11ba92aac765 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:32.487547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:32.487547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:32.487867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:32.488082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] PATH_INFO: `/v3/projects/ee286cfb02ba4b8f9dd748550965a57b` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:32.489420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee286cfb02ba4b8f9dd748550965a57b)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:32.491737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:32.615146 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee286cfb02ba4b8f9dd748550965a57b', 'request_id': 'req-ec65d1f5-3b33-47d5-9114-5c3b112353fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.718484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee286cfb02ba4b8f9dd748550965a57b', 'request_id': 'req-ec65d1f5-3b33-47d5-9114-5c3b112353fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.734841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee286cfb02ba4b8f9dd748550965a57b', 'request_id': 'req-ec65d1f5-3b33-47d5-9114-5c3b112353fa'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.758480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] The token cache is being invalidate because project ee286cfb02ba4b8f9dd748550965a57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:32.758790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.759548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:32.761325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 933/1903] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:32 2022] DELETE /identity/v3/projects/ee286cfb02ba4b8f9dd748550965a57b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:32.765735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:32.776880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b05fd4-5038-42a9-8d5e-d2fe25d08d3d service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:32.844384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:32.846314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:32.846849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:32.847285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] PATH_INFO: `/v3/users/0bb94d5ce6c6477ab35934820950093d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:32.848726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb94d5ce6c6477ab35934820950093d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:32.850934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:33.036171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb94d5ce6c6477ab35934820950093d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.036995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb94d5ce6c6477ab35934820950093d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.051733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb94d5ce6c6477ab35934820950093d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.062674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb94d5ce6c6477ab35934820950093d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.063472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d71487e2-d16f-4b29-b63c-e47333584938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb94d5ce6c6477ab35934820950093d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.152554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 971/1904] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:32 2022] DELETE /identity/v3/users/0bb94d5ce6c6477ab35934820950093d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:33.256911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:33.259376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:33.293220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65d1f5-3b33-47d5-9114-5c3b112353fa admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:33.366592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:33.367637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.367751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.368022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] PATH_INFO: `/v3/projects/64173a599d434a8c95a91276baac03aa` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.369421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64173a599d434a8c95a91276baac03aa)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:33.371506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:33.473523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64173a599d434a8c95a91276baac03aa', 'request_id': 'req-94ac06d2-b23a-4759-b035-3cc1a71bf336'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.558032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64173a599d434a8c95a91276baac03aa', 'request_id': 'req-94ac06d2-b23a-4759-b035-3cc1a71bf336'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.566962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64173a599d434a8c95a91276baac03aa', 'request_id': 'req-94ac06d2-b23a-4759-b035-3cc1a71bf336'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.582241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] The token cache is being invalidate because project 64173a599d434a8c95a91276baac03aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:33.582881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.583750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-94ac06d2-b23a-4759-b035-3cc1a71bf336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:33.586612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 934/1905] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:33 2022] DELETE /identity/v3/projects/64173a599d434a8c95a91276baac03aa => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:33.591606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.591606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.595497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.611256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:33.612500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:33.694360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 972/1906] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:33.696760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.697084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.697331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.718064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:33.719321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:33.731193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 935/1907] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:33.735075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:33.745311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3aabe-ed7f-484c-97a0-0f5e4d02ccce None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:33.750286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:33.750971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.751391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.751674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.752417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:33.752726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:33.754519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:33.767588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 973/1908] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:33.771489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:33.781787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc89483-2883-403f-b45f-5eb36fda3c51 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:33.787476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:33.788198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.788482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.788799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.789485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:33.791470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:33.911069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 936/1909] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:33 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:33.915585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:33.927323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8224-e8d7-43b5-a24d-590f9192eef0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:33.989033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:33.992899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:33.993139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:33.993139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:33.993139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:33.993139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:33.993463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:33.993554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.020840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ae9d5-bdfa-4065-ac6d-d94d2d3fea7c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.030188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.031229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.031520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.032095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] PATH_INFO: `/v3/projects/7261f2bda18d40c68261acfda6c136ca` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.034232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7261f2bda18d40c68261acfda6c136ca)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.036687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.067258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 974/1910] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:33 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:34.071263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.082712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd228899-75ae-49f6-a53c-1e8b6e80484f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.088079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.088783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.089104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.089380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.090243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.091876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.104074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 975/1911] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:34.115512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.144653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505fa4-20e2-4f45-9c00-79a49ffb36e5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.152766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7261f2bda18d40c68261acfda6c136ca', 'request_id': 'req-177aad8b-85e2-41ac-8826-39c810f1c3c8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:34.198752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7261f2bda18d40c68261acfda6c136ca', 'request_id': 'req-177aad8b-85e2-41ac-8826-39c810f1c3c8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:34.207109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7261f2bda18d40c68261acfda6c136ca', 'request_id': 'req-177aad8b-85e2-41ac-8826-39c810f1c3c8'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:34.226431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.227125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.227414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.227694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.228438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.228914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] The token cache is being invalidate because project 7261f2bda18d40c68261acfda6c136ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:34.229342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:34.230181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.230652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-177aad8b-85e2-41ac-8826-39c810f1c3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:34.234427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 937/1912] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:33 2022] DELETE /identity/v3/projects/7261f2bda18d40c68261acfda6c136ca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:34.241317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3a0b-b17b-40bb-aa9e-8cfe999df086 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.241787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3a0b-b17b-40bb-aa9e-8cfe999df086 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.242183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3a0b-b17b-40bb-aa9e-8cfe999df086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.250898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 976/1913] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:34.256210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.259937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:34.276392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168201ab-87e3-4b6a-b199-698612a49946 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.283522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.285098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.285487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.285829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] PATH_INFO: `/v3/projects/c035a2bfe3254762bef72bde2d4d5280/users/7a1ae14cb6434164813e6de515304d1d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.287620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-049d3a0b-b17b-40bb-aa9e-8cfe999df086 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:34.288897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-049d3a0b-b17b-40bb-aa9e-8cfe999df086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:34.306013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 938/1914] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:34.308190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c035a2bfe3254762bef72bde2d4d5280, user_id=7a1ae14cb6434164813e6de515304d1d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.312096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8130e-b703-4df2-9439-2b0b8bad2fcc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.313560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.314588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.314870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.340392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:34.341599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:34.350233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 939/1915] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:34.354246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.366040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932f054-9153-4d97-a263-7fd1add16fed None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.370918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.371582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.371864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.372141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.372866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:34.373175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.374796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.390949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 940/1916] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:34.393437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.404674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ee55-caac-4ab1-aea1-7d020c86ef30 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.410534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 977/1917] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:34 2022] PUT /identity/v3/projects/c035a2bfe3254762bef72bde2d4d5280/users/7a1ae14cb6434164813e6de515304d1d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:34.422181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.422849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.422849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.453004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] MFA Rules not processed for user `7a1ae14cb6434164813e6de515304d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:34.454975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:34.490554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.491380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.491670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.492041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.492858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.495151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.534371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 978/1918] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:34.537523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.550799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633c21e-f1d8-4881-8b53-5592d667e0f9 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.555872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.556595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.556927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.557225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.558103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.560198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.598954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 941/1919] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:34.603324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.667028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e4e2e-6270-4bc7-a19b-bc9670826996 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.676876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.676876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.676876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.676876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.677282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.677625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.684202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 979/1920] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:34.685107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.751617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ab7ee-2c8e-4a91-b22d-a104ac3047a0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.757337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.758183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.759163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.759666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.760614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.762925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.770309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 942/1921] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:34.775351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.783081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b59b68-4224-4366-9301-4678becd0969 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.788826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.789757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.790078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.790397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.791172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.792834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.804585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 943/1922] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:34.808745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.819077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349788ed-e5e4-4d37-9a4c-3b4fe2f0b95b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.823713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.824366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.824646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.824917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.825633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.827399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.839552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 944/1923] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:34.843633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.854156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ebe10-a29b-4e4f-bd9e-f7624c72e0f6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.857110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 980/1924] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:34.861461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.861679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.861932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.862230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] PATH_INFO: `/v3/projects/7875a88d2f584f139cb19d91e1202704/users/24433fcc2f244a7bb18289993b7aa431/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.864344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.877901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1abf32-959b-4b80-b076-9e053d8aa10d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.880538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7875a88d2f584f139cb19d91e1202704, user_id=24433fcc2f244a7bb18289993b7aa431, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.883519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700cfbd-663b-45c1-81de-b39757cf09fe admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.884739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.885758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.886225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.886650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.888171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.889785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.903675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 981/1925] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:34.908225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.920818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c36f49-bcd3-4cac-9de6-8ebbe45fbb4b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.927178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.927930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.928381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.928673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.929828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.931389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.943924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 982/1926] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:34.947696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:34.961829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dacb7-6434-4fea-8c58-7410e70780a3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:34.966627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:34.967514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.967834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.968141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] PATH_INFO: `/v3/projects/e697c82351e94846bc46b06cc67bc64c/users/55fba50702734d7780ba50eedec9e245/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:34.981839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 945/1927] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:34 2022] PUT /identity/v3/projects/7875a88d2f584f139cb19d91e1202704/users/24433fcc2f244a7bb18289993b7aa431/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:34.984532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e697c82351e94846bc46b06cc67bc64c, user_id=55fba50702734d7780ba50eedec9e245, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:34.988079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e9d4-478c-4080-b3a4-e467a8fbf776 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:34.993826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:34.994122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:34.994453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.021078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] MFA Rules not processed for user `24433fcc2f244a7bb18289993b7aa431`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:35.022320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:35.094457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 946/1928] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:35.099428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.107560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 983/1929] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:34 2022] PUT /identity/v3/projects/e697c82351e94846bc46b06cc67bc64c/users/55fba50702734d7780ba50eedec9e245/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:35.119227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.121084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.121373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.149683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] MFA Rules not processed for user `55fba50702734d7780ba50eedec9e245`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:35.150936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:35.162720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb820c86-d0b1-44cf-a78e-a94cd2a20b51 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.218092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 984/1930] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:35.222806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.223604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.224375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.224757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.225160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.225940 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.227975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.233096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261cfcc-37c0-40fb-bd4a-e7c0a086abe1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.238892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.239549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.239827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.240108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.240788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.242732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.302803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 947/1931] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:35.321243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.342496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 985/1932] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:35.349824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.386575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb9048-430c-4bf1-b1bf-2b5b34809ad6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.412848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce7dd-e474-457c-901c-a59dff2499e2 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.417618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.418408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.418765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.419134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.419933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.422039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.450158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.450868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.451154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.451434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.452126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.454140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.504015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 986/1933] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:35.507802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.519574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147730d-70a6-4072-ad8e-af8431be76e5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.528540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.529309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.529594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.529918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.533839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.533839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.535887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 948/1934] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:35.542520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.553526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 987/1935] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:35.553758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7707-ac13-47f2-817f-959d3e612a23 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.554592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.559400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.560429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.561334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.561622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.562398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.564756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.566405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366194d9-d8b5-455b-88ea-eeaac792035b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.571131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.572088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.572560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.572849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] PATH_INFO: `/v3/projects/61559d6dfe314294883e481ec7b35778/users/2e7c597d830f4e4ba67f3fffd092d4ba/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.577182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 949/1936] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:35.581368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.591802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234e338-ce44-48b1-8944-9a70759e3d82 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.592415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61559d6dfe314294883e481ec7b35778, user_id=2e7c597d830f4e4ba67f3fffd092d4ba, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.595975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.597729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.598415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.598704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.599696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.600111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.601690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.614009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 950/1937] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:17:35.620756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.629361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbea6af-a893-477f-9927-bd02ab31eb92 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.634275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.635171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.635450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.635723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] PATH_INFO: `/v3/projects/179baa48e0cf402b80c5029fc9a6923e/users/e7fc0ef014bf4e6c80a95cb03672f6f8/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.671986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179baa48e0cf402b80c5029fc9a6923e, user_id=e7fc0ef014bf4e6c80a95cb03672f6f8, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.675006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804d48-f280-4c97-ab72-c0869e89d099 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.716752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 988/1938] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:35 2022] PUT /identity/v3/projects/61559d6dfe314294883e481ec7b35778/users/2e7c597d830f4e4ba67f3fffd092d4ba/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:35.717623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.776805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 951/1939] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:35 2022] PUT /identity/v3/projects/179baa48e0cf402b80c5029fc9a6923e/users/e7fc0ef014bf4e6c80a95cb03672f6f8/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:35.785119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd45b-ec14-4d77-ba57-dc14a480e475 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.789460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82633-49eb-457f-b709-1e38358f619e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.790571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82633-49eb-457f-b709-1e38358f619e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.790932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b82633-49eb-457f-b709-1e38358f619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.826170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-45b82633-49eb-457f-b709-1e38358f619e None None] MFA Rules not processed for user `e7fc0ef014bf4e6c80a95cb03672f6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:35.828263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-45b82633-49eb-457f-b709-1e38358f619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:35.856455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.857148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.857755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.857755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.858650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.860396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.873448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 989/1940] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:35.878862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.896008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188a47-c14a-4470-9dba-a04b851d9a50 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:35.902925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:35.904219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.904664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.905141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] PATH_INFO: `/v3/projects/61559d6dfe314294883e481ec7b35778/users/2e7c597d830f4e4ba67f3fffd092d4ba/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.908213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61559d6dfe314294883e481ec7b35778, user_id=2e7c597d830f4e4ba67f3fffd092d4ba, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:35.911500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:35.930267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 990/1941] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:35 2022] PUT /identity/v3/projects/61559d6dfe314294883e481ec7b35778/users/2e7c597d830f4e4ba67f3fffd092d4ba/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:35.934642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 952/1942] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:35.939769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:35.940925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:35.941454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:35.941883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:35.988830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] MFA Rules not processed for user `2e7c597d830f4e4ba67f3fffd092d4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:35.996269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:36.042750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eee46-8456-4d96-b256-115102d79818 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.049073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.050790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.051338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.052822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.054807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.056327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.085194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 953/1943] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:36.100480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.101857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:36.148213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 991/1944] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:35 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:36.152098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.179266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49815-14f0-4101-a0a8-260cde0fb879 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.227235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b4cbc-5f86-45be-8eaf-6d01c0b78ef3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.232538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.233219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.233505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.233927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.234650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.237413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.240275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.241028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.241407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.241753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.302434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.304089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.314631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 954/1945] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:36.355654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 992/1946] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:36 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:17:36.359778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.372938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ffa5b-b75d-4d64-8e54-76c7f6534b53 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.378167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.378881 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.379167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.379445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.380172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.381780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.393041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 955/1947] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:36.397369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.408706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22826594-5293-4732-9be4-c74ee8f5225b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.433221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.434935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.435706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.436138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] PATH_INFO: `/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00/users/4f96fb0df55c4c8588a577821bca2dbc/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.457749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d3a434e25c44dfbf4cb6e7313fbc00, user_id=4f96fb0df55c4c8588a577821bca2dbc, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.460443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.541380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 993/1948] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:36 2022] PUT /identity/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00/users/4f96fb0df55c4c8588a577821bca2dbc/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:36.546057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.556917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd894a6-5f2b-443d-92b5-8fedca75bb22 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.614872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.615555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.615842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.616112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.619746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.619746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c714c-051d-44d2-8339-85fb210917d1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.629417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 956/1949] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:36.633560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.645190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8892143-6272-4018-a223-a37286308b83 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.649930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.650845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.651124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.651395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] PATH_INFO: `/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00/users/4f96fb0df55c4c8588a577821bca2dbc/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.653340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d3a434e25c44dfbf4cb6e7313fbc00, user_id=4f96fb0df55c4c8588a577821bca2dbc, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.657076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.668798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 994/1950] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:36 2022] PUT /identity/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00/users/4f96fb0df55c4c8588a577821bca2dbc/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:36.679160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.679481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.679767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.707811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] MFA Rules not processed for user `4f96fb0df55c4c8588a577821bca2dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:36.709182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:36.791769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 957/1951] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:36.805217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:36.806901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:36.869627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1ce4d-7980-48d8-9563-1edf512c4544 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:36.873485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:36.874140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:36.874460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:36.874740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:36.885493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:36.886964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:36.896894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 995/1952] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:37.017782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:37.028154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f359977-3309-43fb-9bca-2bf70be28777 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:37.034580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:37.035397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:37.035518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:37.036136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:37.100932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:37.102533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:37.113225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 958/1953] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:39.542633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:39.553054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bb16-5604-4ca4-8bdb-030716e5777c service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:39.556844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:39.557525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:39.557807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:39.558083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:39.620687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:39.622379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:39.633504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 996/1954] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:41.351438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:41.362412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67f453-2310-4cd0-ada6-d5b47593c77e service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:41.375723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:41.375723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:41.375723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:41.375723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:41.445194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:41.446681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899415d7-a254-4275-b486-e232f79f923c service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:41.456867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 959/1955] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:42.535283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:42.627167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f04651-b321-4eea-a50a-a2b7bb1abce9 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:42.633796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:42.634764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:42.635042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:42.635507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] PATH_INFO: `/v3/users/24433fcc2f244a7bb18289993b7aa431` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:42.636907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] RBAC: Authorizing `identity:delete_user(user_id=24433fcc2f244a7bb18289993b7aa431)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:42.639092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:42.880154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24433fcc2f244a7bb18289993b7aa431'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:42.880927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24433fcc2f244a7bb18289993b7aa431'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:42.894339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24433fcc2f244a7bb18289993b7aa431'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:42.906973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24433fcc2f244a7bb18289993b7aa431'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:42.908855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24433fcc2f244a7bb18289993b7aa431'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:42.997801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 997/1956] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:42 2022] DELETE /identity/v3/users/24433fcc2f244a7bb18289993b7aa431 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:43.002591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:43.003113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:43.003227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:43.022329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:43.023545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:43.087107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 960/1957] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:43.100247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:43.102140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:43.172143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b08b6-f44c-4a02-a6ef-fcf49d6ab28e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:43.175935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:43.176593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:43.176871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:43.177142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:43.186781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:43.188241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:43.198814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 998/1958] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:43.807225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:43.808851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:43.824954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420be67-d41d-49e2-aad9-dc116c94b1cf None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:43.831101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:43.832132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:43.832260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:43.832523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] PATH_INFO: `/v3/projects/7875a88d2f584f139cb19d91e1202704` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:43.834387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7875a88d2f584f139cb19d91e1202704)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:43.836359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:43.918817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7875a88d2f584f139cb19d91e1202704', 'request_id': 'req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:43.981009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7875a88d2f584f139cb19d91e1202704', 'request_id': 'req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:43.987601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7875a88d2f584f139cb19d91e1202704', 'request_id': 'req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:43.999240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] The token cache is being invalidate because project 7875a88d2f584f139cb19d91e1202704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:43.999532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.000201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.002369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 961/1959] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:43 2022] DELETE /identity/v3/projects/7875a88d2f584f139cb19d91e1202704 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:44.007082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:44.017736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64f4db-3e93-40b0-8372-1d5e74acb4fa service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:44.074852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:44.075740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:44.076023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:44.076300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] PATH_INFO: `/v3/users/e7fc0ef014bf4e6c80a95cb03672f6f8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:44.077505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fc0ef014bf4e6c80a95cb03672f6f8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:44.080125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:44.238220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fc0ef014bf4e6c80a95cb03672f6f8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.239035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fc0ef014bf4e6c80a95cb03672f6f8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.254900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fc0ef014bf4e6c80a95cb03672f6f8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.262135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fc0ef014bf4e6c80a95cb03672f6f8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.262908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fc0ef014bf4e6c80a95cb03672f6f8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:44.319812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 999/1960] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:44 2022] DELETE /identity/v3/users/e7fc0ef014bf4e6c80a95cb03672f6f8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:44.847128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:44.849183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:44.865427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec9d58-0fda-4e6e-8042-da6a09c4ae62 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:44.929356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:44.930232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:44.930517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:44.930792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] PATH_INFO: `/v3/projects/179baa48e0cf402b80c5029fc9a6923e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:44.931966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=179baa48e0cf402b80c5029fc9a6923e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:44.933926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:45.021060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179baa48e0cf402b80c5029fc9a6923e', 'request_id': 'req-2d0ae179-db6b-479c-9813-67d5b35d7f5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.095015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179baa48e0cf402b80c5029fc9a6923e', 'request_id': 'req-2d0ae179-db6b-479c-9813-67d5b35d7f5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.101087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179baa48e0cf402b80c5029fc9a6923e', 'request_id': 'req-2d0ae179-db6b-479c-9813-67d5b35d7f5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.112481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] The token cache is being invalidate because project 179baa48e0cf402b80c5029fc9a6923e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:45.112769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.113411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.114994 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 962/1961] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:44 2022] DELETE /identity/v3/projects/179baa48e0cf402b80c5029fc9a6923e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:45.119136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:45.120603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:45.134962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d545b-dce1-42ca-9137-ad1d1dc0eea6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:45.190355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:45.191250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:45.191529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:45.191799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] PATH_INFO: `/v3/users/4f96fb0df55c4c8588a577821bca2dbc` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:45.193039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f96fb0df55c4c8588a577821bca2dbc)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:45.195053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:45.373206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f96fb0df55c4c8588a577821bca2dbc'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.373871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f96fb0df55c4c8588a577821bca2dbc'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.385959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f96fb0df55c4c8588a577821bca2dbc'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.393675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f96fb0df55c4c8588a577821bca2dbc'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.394440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5dfe286b-f004-4243-bf7a-ed83f1e40ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f96fb0df55c4c8588a577821bca2dbc'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:45.490326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1000/1962] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:45 2022] DELETE /identity/v3/users/4f96fb0df55c4c8588a577821bca2dbc => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:45.994742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:45.996287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:46.023259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae179-db6b-479c-9813-67d5b35d7f5e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:46.089132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:46.090687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.090973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.091249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] PATH_INFO: `/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.092481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d3a434e25c44dfbf4cb6e7313fbc00)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:46.094432 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:46.217239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d3a434e25c44dfbf4cb6e7313fbc00', 'request_id': 'req-7039762d-4346-408e-a5a3-2e275a394e5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:46.313347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d3a434e25c44dfbf4cb6e7313fbc00', 'request_id': 'req-7039762d-4346-408e-a5a3-2e275a394e5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:46.319599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d3a434e25c44dfbf4cb6e7313fbc00', 'request_id': 'req-7039762d-4346-408e-a5a3-2e275a394e5e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:46.331653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] The token cache is being invalidate because project 85d3a434e25c44dfbf4cb6e7313fbc00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:46.331945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:46.332609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7039762d-4346-408e-a5a3-2e275a394e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:46.334172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 963/1963] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:45 2022] DELETE /identity/v3/projects/85d3a434e25c44dfbf4cb6e7313fbc00 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:46.340318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.340804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.340926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.359813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:46.361704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:46.420713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1001/1964] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:46.430445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.430736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.431014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.449309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:46.450580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:46.459068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 964/1965] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:46.464622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:46.476330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9131440-e892-4b62-bf56-badca78b38c8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:46.480915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:46.481574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.481852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.578092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.578092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:46.578092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:46.578092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:46.578092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1002/1966] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:46.579303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:46.591474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e0e2-1206-4005-8aec-2c295e4f3662 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:46.598883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:46.599574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.599859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.600152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.600872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:46.603315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:46.757395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 965/1967] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:46 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:46.765535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:46.776799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcfd7a-6319-4d1a-bbae-2b1b6d141d82 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:46.847300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:46.848151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:46.848476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:46.848793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:46.849528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:46.851580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:47.010348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1003/1968] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:46 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:47.014640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:47.024817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0944fb2-9edf-44bd-9345-ccdd46603692 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:47.029492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:47.030152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.030464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.030741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.031492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:47.034266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:47.044856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 966/1969] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:47.052102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:47.062535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713de309-7549-47db-8013-da4b9d68c18f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:47.067871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:47.068752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.069101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.069620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.070461 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:47.072123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec512f7-c226-46ee-aa03-c673e8ab55c6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:47.083657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1004/1970] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:47.089335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:47.100505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d6a1a-2738-484c-bb9f-d4d9d34bde72 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:47.105454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:47.106396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.106684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.106957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] PATH_INFO: `/v3/projects/e1b8882786604d97a6b3a93bd54f91fc/users/64a5376f649345918bc8372676c3b3ff/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.121669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b8882786604d97a6b3a93bd54f91fc, user_id=64a5376f649345918bc8372676c3b3ff, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:47.124611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:47.250194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 967/1971] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:47 2022] PUT /identity/v3/projects/e1b8882786604d97a6b3a93bd54f91fc/users/64a5376f649345918bc8372676c3b3ff/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:47.253679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.253679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.253679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.284193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:47.284964 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:47.364042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1005/1972] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:47.395041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:47.397488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:47.492335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1107fa9-c215-455d-a3de-e537ad354dea admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:47.496388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:47.497130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.497412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.497690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.511247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:47.513018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c35207-9410-43ee-8716-830d14f306b3 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:47.525451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 968/1973] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:47.871087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:47.872878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:47.885989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031684f-b4bd-40f3-8101-6eae77a22ee6 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:47.891202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:47.892608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:47.892887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:47.893160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] PATH_INFO: `/v3/users/7a1ae14cb6434164813e6de515304d1d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:47.894688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1ae14cb6434164813e6de515304d1d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:47.896504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:48.063397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1ae14cb6434164813e6de515304d1d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:48.064105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1ae14cb6434164813e6de515304d1d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:48.076297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1ae14cb6434164813e6de515304d1d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:48.085217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1ae14cb6434164813e6de515304d1d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:48.086475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1ae14cb6434164813e6de515304d1d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:48.211568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1006/1974] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:47 2022] DELETE /identity/v3/users/7a1ae14cb6434164813e6de515304d1d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:48.216034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:48.216482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:48.216595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:48.234305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:48.235543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:48.307806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 969/1975] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:48.319203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:48.320698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:48.381897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830af9-b176-4cc2-9441-235a34c2f6b9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:48.386579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:48.387237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:48.387520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:48.387799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:48.398086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:48.399731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef7af5-43bf-4e5b-aefa-1ddcbd8d9175 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:48.410517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1007/1976] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:49.058448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:49.060441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:49.076659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28489cb-729c-4886-8076-2edfb1241117 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:49.085192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:49.086130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:49.088301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:49.088301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] PATH_INFO: `/v3/projects/c035a2bfe3254762bef72bde2d4d5280` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:49.088715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c035a2bfe3254762bef72bde2d4d5280)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:49.090903 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:49.249241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c035a2bfe3254762bef72bde2d4d5280', 'request_id': 'req-fdea346c-ee43-4764-b423-f6bd80b4b517'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.362031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:49.362442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c035a2bfe3254762bef72bde2d4d5280', 'request_id': 'req-fdea346c-ee43-4764-b423-f6bd80b4b517'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.362578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:49.363057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:49.369776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c035a2bfe3254762bef72bde2d4d5280', 'request_id': 'req-fdea346c-ee43-4764-b423-f6bd80b4b517'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.391450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] The token cache is being invalidate because project c035a2bfe3254762bef72bde2d4d5280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:49.393080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.394114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.396582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 970/1977] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:49 2022] DELETE /identity/v3/projects/c035a2bfe3254762bef72bde2d4d5280 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:17:49.400907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:49.402862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:49.406813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] MFA Rules not processed for user `64a5376f649345918bc8372676c3b3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:49.409701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:49.504047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea346c-ee43-4764-b423-f6bd80b4b517 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:49.510785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:49.511622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:49.511919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:49.512201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] PATH_INFO: `/v3/users/55fba50702734d7780ba50eedec9e245` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:49.513805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fba50702734d7780ba50eedec9e245)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:49.515825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:49.518744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1008/1978] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:49.519855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:49.521240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:49.541359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4d18-cb43-4fbd-a4b9-ff4a4e35840e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:49.546591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:49.547410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:49.547676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:49.547957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:49.550128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:49.553234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:49.698960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1009/1979] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:49 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:49.708520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:49.786491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81c7e4-86c8-4520-a1db-d85eca05baf9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:49.792479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:49.793545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:49.793837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:49.794114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:49.794907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:49.798126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:49.803689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fba50702734d7780ba50eedec9e245'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.804504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fba50702734d7780ba50eedec9e245'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.818635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fba50702734d7780ba50eedec9e245'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.820972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fba50702734d7780ba50eedec9e245'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:49.828145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fba50702734d7780ba50eedec9e245'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:50.032482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1010/1980] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:49 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:50.034962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:50.041333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:50.056246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af622-033c-424d-a883-70cfa5445590 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:50.061744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:50.064655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:50.064655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:50.064655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:50.064655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:50.066471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:50.085293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1011/1981] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:50.087069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:50.100471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 971/1982] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:49 2022] DELETE /identity/v3/users/55fba50702734d7780ba50eedec9e245 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:50.101514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a33611-2140-4650-a430-8ebfcf967a75 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:50.187011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:50.187934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:50.188219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:50.188496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] PATH_INFO: `/v3/projects/bcbf128b5c46470782789b15b1cbb968/users/e52a8d3dc96a4766b9e17391a10c8f55/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:50.207027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbf128b5c46470782789b15b1cbb968, user_id=e52a8d3dc96a4766b9e17391a10c8f55, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:50.212172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:50.330397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1012/1983] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:50 2022] PUT /identity/v3/projects/bcbf128b5c46470782789b15b1cbb968/users/e52a8d3dc96a4766b9e17391a10c8f55/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:50.334397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:50.344626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4171ab-e6f4-4fb7-837f-6b98def23280 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:50.406945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:50.407973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:50.408441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:50.408766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:50.409658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:50.411329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:50.422322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 972/1984] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:50.427211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:50.443639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebb548-7607-4f37-91be-8dd90a537aa2 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:50.449157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:50.450120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:50.450433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:50.450743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] PATH_INFO: `/v3/projects/bcbf128b5c46470782789b15b1cbb968/users/e52a8d3dc96a4766b9e17391a10c8f55/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:50.453196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbf128b5c46470782789b15b1cbb968, user_id=e52a8d3dc96a4766b9e17391a10c8f55, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:50.456136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:50.467232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1013/1985] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:50 2022] PUT /identity/v3/projects/bcbf128b5c46470782789b15b1cbb968/users/e52a8d3dc96a4766b9e17391a10c8f55/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:50.641528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:50.645830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:50.683191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5415-fe79-4ae6-81ed-51b0516ac92e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:50.762085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:50.763319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:50.763611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:50.763889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] PATH_INFO: `/v3/projects/e697c82351e94846bc46b06cc67bc64c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:50.765114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e697c82351e94846bc46b06cc67bc64c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:50.768388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:50.897086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e697c82351e94846bc46b06cc67bc64c', 'request_id': 'req-768418f6-ebf5-455d-8f98-5d37318ae316'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.027321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e697c82351e94846bc46b06cc67bc64c', 'request_id': 'req-768418f6-ebf5-455d-8f98-5d37318ae316'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.034478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e697c82351e94846bc46b06cc67bc64c', 'request_id': 'req-768418f6-ebf5-455d-8f98-5d37318ae316'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.045444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] The token cache is being invalidate because project e697c82351e94846bc46b06cc67bc64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:51.045734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.046403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.047970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 973/1986] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:50 2022] DELETE /identity/v3/projects/e697c82351e94846bc46b06cc67bc64c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:51.052422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:51.055389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:51.068514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297a48f-a154-4dad-8f6b-dc76f039858f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:51.139714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:51.139714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:51.139714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:51.139714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] PATH_INFO: `/v3/users/2e7c597d830f4e4ba67f3fffd092d4ba` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:51.140066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7c597d830f4e4ba67f3fffd092d4ba)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:51.140066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:51.366278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7c597d830f4e4ba67f3fffd092d4ba'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.366947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7c597d830f4e4ba67f3fffd092d4ba'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.379594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7c597d830f4e4ba67f3fffd092d4ba'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.386776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7c597d830f4e4ba67f3fffd092d4ba'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.387496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-40b9fc2d-9647-41ae-8e0e-d10e8c39a16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7c597d830f4e4ba67f3fffd092d4ba'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:51.495875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1014/1987] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:17:51 2022] DELETE /identity/v3/users/2e7c597d830f4e4ba67f3fffd092d4ba => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:52.566829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:52.577797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768418f6-ebf5-455d-8f98-5d37318ae316 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:52.642975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:52.643907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:52.644211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:52.644488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] PATH_INFO: `/v3/projects/61559d6dfe314294883e481ec7b35778` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:52.645769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61559d6dfe314294883e481ec7b35778)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:52.647727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:52.686725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:52.687179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:52.687415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:52.716477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] MFA Rules not processed for user `e52a8d3dc96a4766b9e17391a10c8f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:52.717890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:52.803507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1015/1988] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:52.811978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61559d6dfe314294883e481ec7b35778', 'request_id': 'req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:52.816147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:52.817691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:52.875970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15be8f-92f6-43e6-b44a-fca0e9f06683 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:52.901705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61559d6dfe314294883e481ec7b35778', 'request_id': 'req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:52.907620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61559d6dfe314294883e481ec7b35778', 'request_id': 'req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:52.919291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] The token cache is being invalidate because project 61559d6dfe314294883e481ec7b35778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:17:52.919584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:52.920242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b4a867f3-6ffa-4cbf-a1ad-2556d82af856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:17:52.921925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 974/1989] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:17:52 2022] DELETE /identity/v3/projects/61559d6dfe314294883e481ec7b35778 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:52.926522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:52.932942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:52.932942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:52.932942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:52.932942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:52.936500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:52.936863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:52.937219 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:52.956510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:52.957765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:52.990963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:52.992410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ccaf5-e74b-41f8-aa9c-dbe5bd0cffff service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.002664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1016/1990] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:53.016697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 975/1991] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:17:53.026230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.026524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.026807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.051821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:53.053107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:53.061359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1017/1992] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:53.065949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.075476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea24321-ee6c-4bdd-b7f6-4e3d11cfcfaf None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.081326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.081993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.082307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.086843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.086843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:17:53.086843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.086843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.097832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 976/1993] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:17:53.102118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.112647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca6708-c4c3-44ec-868a-a6928b0f22e5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.117670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.118512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.118869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.119225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.120017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.122090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.236406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1018/1994] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:17:53 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:53.240668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.250889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9261-b143-473b-a44f-14e9a2a3b91a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.310412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.311172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.311521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.311892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.312658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.314899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.428988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 977/1995] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:17:53 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:17:53.430257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.442308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401274a-543d-4469-b4e8-93599062be5d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.447532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.448176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.448456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.448797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.449525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.451261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.462140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1019/1996] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:53.466325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.478549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adf4a-6af1-4dfa-bfa1-40b8dcdc61de admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.483341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.484052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.484338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.484617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.485347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.487902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758a1de-f643-4fbb-87f8-86e4d89da863 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.498358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 978/1997] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:17:53.503534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.514163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e35f1-085b-4bce-893b-96738c858c57 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.520360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.521350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.521640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.521912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] PATH_INFO: `/v3/projects/709b9712f09a42a785d046e0c0f3846e/users/48efa16cb9844fcd853f1e3760619a66/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.537380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=709b9712f09a42a785d046e0c0f3846e, user_id=48efa16cb9844fcd853f1e3760619a66, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.540297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.663829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1020/1998] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:17:53 2022] PUT /identity/v3/projects/709b9712f09a42a785d046e0c0f3846e/users/48efa16cb9844fcd853f1e3760619a66/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:17:53.669282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10684a27-4142-4aae-8e34-efc552ab30e5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.669795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10684a27-4142-4aae-8e34-efc552ab30e5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.669970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10684a27-4142-4aae-8e34-efc552ab30e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.690356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-10684a27-4142-4aae-8e34-efc552ab30e5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:53.691458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-10684a27-4142-4aae-8e34-efc552ab30e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:53.755560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 979/1999] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:53.770341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:53.770341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:53.825850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a542-6e01-4812-a504-bfe6ae25ec7f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:53.829544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:53.830770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:53.831059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:53.831337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:53.840716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:53.842171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:53.851874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1021/2000] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:17:55.891327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:55.895284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:55.895284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:55.924722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] MFA Rules not processed for user `48efa16cb9844fcd853f1e3760619a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:55.926021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:55.948996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:55.950463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:56.012398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 980/2001] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:56.018087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50f011-cd0c-4586-930e-309a4f982c15 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:56.021118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:17:56.022649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:56.028760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:56.029903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:56.030410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:56.030904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:56.033998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:56.045747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f5dc-8153-44cf-8b5e-adbd52dcda07 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:17:56.050794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:17:56.051464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:56.051750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:56.052307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:56.062833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:56.064864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c189012-8d7e-4917-9769-cfe59d816b46 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:56.075236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 981/2002] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:56.127760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:17:56.129251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a380e10-4fdc-4f7f-b19d-4c8d2f463668 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:17:56.150730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1022/2003] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:17:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:17:57.498598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:57.499108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:57.499198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:57.517749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] MFA Rules not processed for user `7669f2f83550428893d6c91614b78f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:57.519277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:57.577132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 982/2004] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:17:58.883257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:17:58.883802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:17:58.883930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:17:58.903020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] MFA Rules not processed for user `48efa16cb9844fcd853f1e3760619a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:17:58.904225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:17:58.911539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1023/2005] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:17:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:00.598589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:00.600201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:00.614411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13199e14-2222-4af1-8d1b-40320a933636 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:00.620110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:00.621103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:00.621385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:00.621659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] PATH_INFO: `/v3/users/f00f343268134dea89e0a7070da14508` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:00.622894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00f343268134dea89e0a7070da14508)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:00.628513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:00.919572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00f343268134dea89e0a7070da14508'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:00.920408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00f343268134dea89e0a7070da14508'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:00.940355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00f343268134dea89e0a7070da14508'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:00.942003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00f343268134dea89e0a7070da14508'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:00.960367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-103ef215-47c5-427f-9a66-de4105fc513f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00f343268134dea89e0a7070da14508'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.033834 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 983/2006] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:00 2022] DELETE /identity/v3/users/f00f343268134dea89e0a7070da14508 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:01.293705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:01.327472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c6aa-59cb-4e5d-8639-e0c7d50de0d7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:01.380976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:01.381851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:01.382126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:01.382430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] PATH_INFO: `/v3/projects/6e0f048425c440288655a28aecc12ce5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:01.383662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0f048425c440288655a28aecc12ce5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:01.385568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:01.510928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0f048425c440288655a28aecc12ce5', 'request_id': 'req-70a5302c-d1e8-4d07-a04b-22dbc56848c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.598069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0f048425c440288655a28aecc12ce5', 'request_id': 'req-70a5302c-d1e8-4d07-a04b-22dbc56848c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.606743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0f048425c440288655a28aecc12ce5', 'request_id': 'req-70a5302c-d1e8-4d07-a04b-22dbc56848c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.618741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] The token cache is being invalidate because project 6e0f048425c440288655a28aecc12ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:01.619029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.619721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-70a5302c-d1e8-4d07-a04b-22dbc56848c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:01.621518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1024/2007] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:01 2022] DELETE /identity/v3/projects/6e0f048425c440288655a28aecc12ce5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:01.628511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:01.633771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:01.633771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:01.648021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:01.649246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:01.709873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 984/2008] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:18:01.717684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:01.718007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:01.718357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:01.737824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:01.739068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:01.747048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1025/2009] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:01.751227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:01.765652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b5ee6-a0dd-435c-a1a5-d666f9f62923 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:01.766655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:01.767838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:01.768125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:01.768519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:01.769824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:18:01.769824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:01.771179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:01.786240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 985/2010] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:18:01.794402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:01.804970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247da7-8765-4788-a5ce-abdd58c84099 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:01.809724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:01.810488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:01.810814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:01.811378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:01.812081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:01.814391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:01.943660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1026/2011] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:01 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:01.950888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:01.964321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ce4a-77fa-4a4e-8572-d6194616e46d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.026842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.027530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.027823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.028104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.028801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.030773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.139782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 986/2012] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:01 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:02.142708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.162565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4631b27-43cb-4213-923a-731d079ac944 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.165898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.166673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.167018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.167470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.168211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.169952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.181917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1027/2013] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:02.186106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.196304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11124b52-e939-460c-a58b-a3147f246542 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.200957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.201621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.201902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.202170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.202966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.204550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accfe4f-ea64-4459-88c9-10f87768644d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.214900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 987/2014] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:18:02.219176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.229954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6a1e-44fd-45a2-8a89-fda5a1596273 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.234529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.237046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.237329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.237601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] PATH_INFO: `/v3/projects/aad1955dbe4f4619832510cd50d6deba/users/399ab9ecaad84332b2153f462aacf47d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.253426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad1955dbe4f4619832510cd50d6deba, user_id=399ab9ecaad84332b2153f462aacf47d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.256121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.339699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1028/2015] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:02 2022] PUT /identity/v3/projects/aad1955dbe4f4619832510cd50d6deba/users/399ab9ecaad84332b2153f462aacf47d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:02.353048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.353151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.353422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.389199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] MFA Rules not processed for user `399ab9ecaad84332b2153f462aacf47d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:02.389852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:02.466181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 988/2016] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:18:02.471987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.540999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03be0ff-ab6e-4428-8e94-ac049a315d99 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.546271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.549216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.549930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.550399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.551294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.553396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.623381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1029/2017] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:02.628961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.641698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ab74-4885-4a35-971e-f8cfaa2c21d7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.712007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.712928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.713214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.713491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.714335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.716368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.803243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 989/2018] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:02 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:02.807545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.819555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d01f6-89df-46dc-a0b0-6ce9a069afbd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.825140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.826001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.826476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.826855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.827734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.829615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.843609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1030/2019] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:02.849125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.867502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b867e-6409-4732-b7d9-a0c386752d18 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.872551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.873214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.873491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.874218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.874739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.876121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138c564-e7c7-42bb-bc16-bf3a31e3a43f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.890510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 990/2020] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:02.891919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:02.909143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d47fe-17ff-43b5-820c-84d32df4e2d4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:02.915419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:02.916595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:02.916595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:02.916875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] PATH_INFO: `/v3/projects/0bd8e6af77e541729a2e26827d16114b/users/1015395da018408590cdf481737b44ed/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:02.934443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd8e6af77e541729a2e26827d16114b, user_id=1015395da018408590cdf481737b44ed, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:02.939765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:02.994989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1031/2021] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:02 2022] PUT /identity/v3/projects/0bd8e6af77e541729a2e26827d16114b/users/1015395da018408590cdf481737b44ed/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:03.006830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.007135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.007412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.046935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] MFA Rules not processed for user `1015395da018408590cdf481737b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:03.048312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:03.121565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 991/2022] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:03.122416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.196581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b63c1-3923-4a58-801b-6b139a9ba884 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.202155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.203064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.210913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.210913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.210913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.210913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.337114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1032/2023] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:03 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:03.339946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.352395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52210e0-3870-466e-b6e7-6d259a082d71 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.414959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.414959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.414959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.414959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.415356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.415356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.505643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 992/2024] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:03 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:03.507446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.519410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244811c-8617-4137-8f66-2117ff26feb3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.524713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.525463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.525819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.526099 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.527040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.528998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.541032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1033/2025] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:03.545408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.557658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763de20-9f2b-48f3-841e-a020d6f1f2b2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.562864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.564277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.564550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.564828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] PATH_INFO: `/v3/projects/e9e95ba92ca043d1b838db0491918b93/users/438ac719a654458b969e94fc41aa4a26/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.582362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e95ba92ca043d1b838db0491918b93, user_id=438ac719a654458b969e94fc41aa4a26, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.585056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.685261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 993/2026] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:03 2022] PUT /identity/v3/projects/e9e95ba92ca043d1b838db0491918b93/users/438ac719a654458b969e94fc41aa4a26/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:03.689614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.700137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c8016-2a5b-4d7e-9244-1e477c70579b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.760001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.767097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.767097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.767097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.767097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.767097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7a45c-a98c-4517-a895-fbb7f1b16aa5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.775573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1034/2027] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:18:03.779797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.792119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e143-bdb1-4cd0-8e02-f127bc0b9cdd admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:03.797186 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:03.798155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.799231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.799625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] PATH_INFO: `/v3/projects/e9e95ba92ca043d1b838db0491918b93/users/438ac719a654458b969e94fc41aa4a26/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.801716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e95ba92ca043d1b838db0491918b93, user_id=438ac719a654458b969e94fc41aa4a26, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:03.804428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:03.815817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 994/2028] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:03 2022] PUT /identity/v3/projects/e9e95ba92ca043d1b838db0491918b93/users/438ac719a654458b969e94fc41aa4a26/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:03.826251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:03.826548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:03.826854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:03.855247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] MFA Rules not processed for user `438ac719a654458b969e94fc41aa4a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:03.856468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:03.937516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1035/2029] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:18:03.950807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:03.952698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:04.052098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf23453-37d4-44d5-ac3c-a822fe3c5a84 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:04.056313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:04.057090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:04.057578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:04.057937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:04.087691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:04.089343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:04.103006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 995/2030] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:05.263295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:05.273596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c824e5-6c9e-47aa-b398-ea21bdfbbd47 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:05.277348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:05.277984 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:05.278294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:05.278569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:05.337588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:05.339100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:05.348594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1036/2031] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:13.812359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:13.813823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:13.861957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb96c-c614-4c67-ab1a-bcdf61c76689 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:13.866010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:13.866768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:13.867057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:13.867337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:13.950490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:13.951963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:13.964134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 996/2032] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:17.843988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:17.948716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07850505-fe58-4a3b-bd3c-63b1448db798 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:17.953590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:17.954491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:17.954777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:17.955142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] PATH_INFO: `/v3/users/399ab9ecaad84332b2153f462aacf47d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:17.956521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399ab9ecaad84332b2153f462aacf47d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:17.958505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:18.116246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399ab9ecaad84332b2153f462aacf47d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.116904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399ab9ecaad84332b2153f462aacf47d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.128352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399ab9ecaad84332b2153f462aacf47d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.135435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399ab9ecaad84332b2153f462aacf47d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.136125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08eab262-9d20-48f1-acad-3e62fbff33b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399ab9ecaad84332b2153f462aacf47d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.145565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:18.147472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.184442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dc752-7ebc-44a1-9eb3-864f74979cf0 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:18.189525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:18.190519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:18.190868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:18.191212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] PATH_INFO: `/v3/users/64a5376f649345918bc8372676c3b3ff` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:18.192506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a5376f649345918bc8372676c3b3ff)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:18.194650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:18.235013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1037/2033] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:17 2022] DELETE /identity/v3/users/399ab9ecaad84332b2153f462aacf47d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:18.238991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:18.239283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:18.239563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:18.266619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:18.268618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.327872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1038/2034] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:18.338357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:18.339748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.344838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a5376f649345918bc8372676c3b3ff'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.345519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a5376f649345918bc8372676c3b3ff'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.358420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a5376f649345918bc8372676c3b3ff'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.359184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a5376f649345918bc8372676c3b3ff'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.365892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a5376f649345918bc8372676c3b3ff'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:18.417551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079c6e3-8a54-4018-87e7-46689f810aae None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:18.421962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:18.422646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:18.422931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:18.423201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:18.426171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.433777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 997/2035] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:18 2022] DELETE /identity/v3/users/64a5376f649345918bc8372676c3b3ff => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:18.438950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:18.440659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:18.450721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1039/2036] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:18:18.848635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:18.850123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.934875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b77e5f-b8de-40be-ac68-f04b5b2a54cf admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:18.951839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:18.952474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:18.952474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:18.952637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:18.952734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:18.952734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] PATH_INFO: `/v3/projects/e1b8882786604d97a6b3a93bd54f91fc` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:18.953211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b8882786604d97a6b3a93bd54f91fc)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:18.956973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:18.964262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35cc13-3afd-46ba-a94d-20022121ac2d service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:18.979543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:18.980430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:18.980752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:18.981199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] PATH_INFO: `/v3/projects/aad1955dbe4f4619832510cd50d6deba` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:18.982887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad1955dbe4f4619832510cd50d6deba)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:18.987318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:19.079891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b8882786604d97a6b3a93bd54f91fc', 'request_id': 'req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.110759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad1955dbe4f4619832510cd50d6deba', 'request_id': 'req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.142285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b8882786604d97a6b3a93bd54f91fc', 'request_id': 'req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.148985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b8882786604d97a6b3a93bd54f91fc', 'request_id': 'req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.161263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] The token cache is being invalidate because project e1b8882786604d97a6b3a93bd54f91fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:19.161557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.162231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.163800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 998/2037] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:18 2022] DELETE /identity/v3/projects/e1b8882786604d97a6b3a93bd54f91fc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:19.167560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:19.168825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:19.204829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad1955dbe4f4619832510cd50d6deba', 'request_id': 'req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.211613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad1955dbe4f4619832510cd50d6deba', 'request_id': 'req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.225057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] The token cache is being invalidate because project aad1955dbe4f4619832510cd50d6deba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:19.225362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.225994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.227840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1040/2038] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:18 2022] DELETE /identity/v3/projects/aad1955dbe4f4619832510cd50d6deba => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:19.231397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:19.233286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:19.234125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09ccf-b66a-41aa-bbab-9a97c3a051cd admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:19.294806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:19.295690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:19.295973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:19.296249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] PATH_INFO: `/v3/users/e52a8d3dc96a4766b9e17391a10c8f55` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:19.297483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52a8d3dc96a4766b9e17391a10c8f55)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:19.299860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:19.303745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0923-6bee-47a5-8e27-2f7b6f1490a6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:19.307571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:19.308465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:19.308746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:19.309012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] PATH_INFO: `/v3/users/1015395da018408590cdf481737b44ed` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:19.310290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1015395da018408590cdf481737b44ed)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:19.312366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:19.452991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52a8d3dc96a4766b9e17391a10c8f55'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.453692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52a8d3dc96a4766b9e17391a10c8f55'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.464414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1015395da018408590cdf481737b44ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.465116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1015395da018408590cdf481737b44ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.466305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52a8d3dc96a4766b9e17391a10c8f55'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.467004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52a8d3dc96a4766b9e17391a10c8f55'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.474397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52a8d3dc96a4766b9e17391a10c8f55'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.476938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1015395da018408590cdf481737b44ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.483629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1015395da018408590cdf481737b44ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.484313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d1b9210e-0805-4b5e-a18f-0954f8c661e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1015395da018408590cdf481737b44ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:19.536873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 999/2039] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:19 2022] DELETE /identity/v3/users/e52a8d3dc96a4766b9e17391a10c8f55 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:19.617526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1041/2040] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:19 2022] DELETE /identity/v3/users/1015395da018408590cdf481737b44ed => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:19.880351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:19.881852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:19.955895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc922-1a4f-49d3-8603-f1a8f5563fca admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.019032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.020110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.020390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.020703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] PATH_INFO: `/v3/projects/bcbf128b5c46470782789b15b1cbb968` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.021958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbf128b5c46470782789b15b1cbb968)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.024101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.137031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbf128b5c46470782789b15b1cbb968', 'request_id': 'req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.223013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbf128b5c46470782789b15b1cbb968', 'request_id': 'req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.229954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbf128b5c46470782789b15b1cbb968', 'request_id': 'req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.242108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] The token cache is being invalidate because project bcbf128b5c46470782789b15b1cbb968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:20.242474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.243188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.244852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1000/2041] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:19 2022] DELETE /identity/v3/projects/bcbf128b5c46470782789b15b1cbb968 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:20.252002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2074-077b-430c-93dc-e0d06723ae92 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.252558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2074-077b-430c-93dc-e0d06723ae92 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.252558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c2074-077b-430c-93dc-e0d06723ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.288999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e60c2074-077b-430c-93dc-e0d06723ae92 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:20.290181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e60c2074-077b-430c-93dc-e0d06723ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:20.321232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.322767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:20.352571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1042/2042] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:20.359209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.359501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.359788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.378295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:20.379500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:20.387514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1043/2043] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:20.388390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6d13d-c762-4dfc-b1cd-cf38494f76ba admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.391574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.393641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.394587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.394874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.395155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] PATH_INFO: `/v3/projects/0bd8e6af77e541729a2e26827d16114b` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.396361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd8e6af77e541729a2e26827d16114b)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.398648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.400369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca599cf-2dd1-4d56-a7aa-bb704be1d687 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.405872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.406576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.406862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.407139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.407869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:18:20.408808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.410893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.423106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1044/2044] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:18:20.427915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.438693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1b3d0-7753-49ee-9ba6-7bcb253332a6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.443519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.444155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.444434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.444702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.445380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.447302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.502967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd8e6af77e541729a2e26827d16114b', 'request_id': 'req-585a873e-570a-4362-b218-e504be0b9989'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.577844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1045/2045] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:20 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:20.582551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.605411 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd8e6af77e541729a2e26827d16114b', 'request_id': 'req-585a873e-570a-4362-b218-e504be0b9989'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.611959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd8e6af77e541729a2e26827d16114b', 'request_id': 'req-585a873e-570a-4362-b218-e504be0b9989'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.623184 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] The token cache is being invalidate because project 0bd8e6af77e541729a2e26827d16114b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:20.623473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.624174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.625755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1001/2046] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:20 2022] DELETE /identity/v3/projects/0bd8e6af77e541729a2e26827d16114b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:20.629982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.631257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:20.644058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8154ba-96ac-40b6-863b-ffab02aa028b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.648938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.649610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.649888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.655454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.655454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.655454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.695896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a873e-570a-4362-b218-e504be0b9989 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.700588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.701487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.701771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.702048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] PATH_INFO: `/v3/users/438ac719a654458b969e94fc41aa4a26` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.703289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438ac719a654458b969e94fc41aa4a26)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.707116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.759975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1046/2047] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:20 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:20.763899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.765472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:20.777965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21caf-67a4-4b08-912f-ef470b46c14e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.782722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.784418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.784955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.785273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.785997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.787620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.797292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1047/2048] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:18:20.801256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.810480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb888ff-ab92-4834-b06e-e840ccaa649f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.815480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.816097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.816377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.816652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.817371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.819009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.828865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1048/2049] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:18:20.833458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:20.842545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db46bb6-7a28-4788-a5d5-3818ae479a42 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:20.848183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:20.848999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.849288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.849570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] PATH_INFO: `/v3/projects/64cdfef13989407893648db00239b75a/users/ef6b1765e6804fe784037b56fb7d6419/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.863954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] RBAC: Authorizing `identity:create_grant(project_id=64cdfef13989407893648db00239b75a, user_id=ef6b1765e6804fe784037b56fb7d6419, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:20.866686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae1115-deff-4651-8b05-4d5276f938db admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:20.881797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438ac719a654458b969e94fc41aa4a26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.882405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438ac719a654458b969e94fc41aa4a26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.894597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438ac719a654458b969e94fc41aa4a26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.895297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438ac719a654458b969e94fc41aa4a26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.901883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438ac719a654458b969e94fc41aa4a26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:20.959460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1049/2050] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:20 2022] PUT /identity/v3/projects/64cdfef13989407893648db00239b75a/users/ef6b1765e6804fe784037b56fb7d6419/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:20.969186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:20.969477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:20.969755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:20.993724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1002/2051] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:20 2022] DELETE /identity/v3/users/438ac719a654458b969e94fc41aa4a26 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:20.999992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] MFA Rules not processed for user `ef6b1765e6804fe784037b56fb7d6419`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:20.999992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:21.062748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1050/2052] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:18:21.074369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:21.075819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:21.132768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e34c-0fe7-4440-91a9-913cafc83e26 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:21.136472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:21.137109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:21.137387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:21.137660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:21.146757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:21.149123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3e507-1e30-45b6-9cdd-aa8cd82e94a4 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:21.156876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1003/2053] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:21.595606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:21.597875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:21.670357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0d54-2f68-4ceb-968b-c7c615eaa2b5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:21.675837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:21.676696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:21.676982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:21.677257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] PATH_INFO: `/v3/projects/e9e95ba92ca043d1b838db0491918b93` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:21.678592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e95ba92ca043d1b838db0491918b93)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:21.680630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:21.831737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e95ba92ca043d1b838db0491918b93', 'request_id': 'req-64897a85-b012-48d5-b883-8c09c44cd04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:21.913823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e95ba92ca043d1b838db0491918b93', 'request_id': 'req-64897a85-b012-48d5-b883-8c09c44cd04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:21.920312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e95ba92ca043d1b838db0491918b93', 'request_id': 'req-64897a85-b012-48d5-b883-8c09c44cd04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:21.932656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] The token cache is being invalidate because project e9e95ba92ca043d1b838db0491918b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:21.932952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:21.933675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-64897a85-b012-48d5-b883-8c09c44cd04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:21.935568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1051/2054] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:21 2022] DELETE /identity/v3/projects/e9e95ba92ca043d1b838db0491918b93 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:21.941497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:21.947252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:21.947252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:21.963877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:21.965405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:22.029382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1004/2055] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:22.039840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.040340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.040475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.061158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:22.062400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:22.071206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1052/2056] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:22.075539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.085076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc912f-0cea-4a2b-b9e5-13a331682cf9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.089768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.090463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.090752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.091031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.091757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:18:22.092131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.093648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.106182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1005/2057] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:18:22.110439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.120143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985cf6-1183-4973-83a4-772e323e46e8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.124834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.125491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.125774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.126068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.126789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.129504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.218146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1053/2058] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:22.222650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.232558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fd6f6-a7b1-4a84-9cfc-6ad71397207a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.295780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.296464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.296752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.297031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.297806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.301203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.388332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1006/2059] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:22.392289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.403462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bf254-daea-4a4a-bad2-95855cc75e7c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.408107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.408797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.409084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.409362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.410397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.412005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.422154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1054/2060] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:22.426365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.436269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e92d-842a-4bd1-b369-65bc1fc5def1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.441712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.442403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.442690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.442993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.443713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.445308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fbf69-c257-482d-9fee-51ece2a7c019 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.456113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1007/2061] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:22.460289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.470739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b74ca-a46f-4155-9cf0-1064118afe17 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.475432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.476300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.476582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.476855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] PATH_INFO: `/v3/projects/60b28373984a47cda134ce8849619220/users/6a80b210d9794c48801202db6278e728/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.492345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b28373984a47cda134ce8849619220, user_id=6a80b210d9794c48801202db6278e728, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.495061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.574457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1055/2062] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:22 2022] PUT /identity/v3/projects/60b28373984a47cda134ce8849619220/users/6a80b210d9794c48801202db6278e728/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:22.586529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.589065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.589065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.637036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] MFA Rules not processed for user `6a80b210d9794c48801202db6278e728`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:22.639084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:22.721735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1008/2063] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:22.725516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.816898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc861-ac46-4793-ae49-5c93ba5ea330 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.821851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:22.822581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:22.822868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:22.827489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:22.827489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:22.827489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:22.925377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1056/2064] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/projects => generated 391 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:22.929731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:22.941258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174f1cc-ef9b-483c-9a86-e9384052e2dc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:22.999536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.004435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.004435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.004435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.004435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.004435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.088722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1009/2065] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:22 2022] POST /identity/v3/users => generated 302 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:23.089924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.100080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aaa1b-bd1a-44c2-b7e3-c46d43f8f6d9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.104698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.105355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.105639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.105915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.106750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.108396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.119382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1057/2066] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:23.123521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.133946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432a132-d513-41be-861b-930a2e2b4688 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.138644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.139610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.139897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.140172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] PATH_INFO: `/v3/projects/0a636dce4e8c47488685088b7e855b80/users/ace26b9c782f4c709cdc2edb5fe2d846/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.154416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a636dce4e8c47488685088b7e855b80, user_id=ace26b9c782f4c709cdc2edb5fe2d846, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.157193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.314572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1010/2067] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:23 2022] PUT /identity/v3/projects/0a636dce4e8c47488685088b7e855b80/users/ace26b9c782f4c709cdc2edb5fe2d846/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:23.318974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.328801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4ab9d-f178-48b4-9681-f8af0d55a4af admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.389477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.390144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.390451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.390721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.391435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.393236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765f7e-b0f8-437d-aeed-3e867caa99cd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.402947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1058/2068] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:23.407445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.418901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812021d-f7f8-4357-bb9d-a20a55a1965d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.423582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.425560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.425841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.426113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] PATH_INFO: `/v3/projects/0a636dce4e8c47488685088b7e855b80/users/ace26b9c782f4c709cdc2edb5fe2d846/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.428082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a636dce4e8c47488685088b7e855b80, user_id=ace26b9c782f4c709cdc2edb5fe2d846, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.430769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.441371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1011/2069] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:23 2022] PUT /identity/v3/projects/0a636dce4e8c47488685088b7e855b80/users/ace26b9c782f4c709cdc2edb5fe2d846/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:23.452730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.453017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.453293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.480142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] MFA Rules not processed for user `ace26b9c782f4c709cdc2edb5fe2d846`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:23.487243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:23.559571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1059/2070] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:23.577007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.578241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:23.646287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb2297-e434-4a90-8351-8dce00768cf7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.650250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.650907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.651194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.651473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.661971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.663599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f08f2d-6d8e-4305-98d7-adec8abaf5c5 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.676499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1012/2071] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:23.731724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:23.733171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:23.745951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebab44-59b3-47d1-9181-f7b82a746d93 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:23.749766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:23.750445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:23.750727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:23.751003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:23.809763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:23.811233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:23.821316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1060/2072] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:47.156100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:47.157827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:47.157827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:47.185273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:47.186483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:47.252141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1013/2073] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:47.298158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:47.299785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:47.325560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:47.372941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afac784-bd94-4aeb-bb92-772f04792be2 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:47.378218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:47.378935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:47.379226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:47.379517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:47.387756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:47.389342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee61d-88d9-4d07-985d-70bfbd0ef663 service swift] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:47.392967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ae18f-6d49-49fe-8ddb-2942771e9c1f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:47.398396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:47.398783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1061/2074] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:18:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:18:47.399278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:47.399617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:47.399928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] PATH_INFO: `/v3/users/6a80b210d9794c48801202db6278e728` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:47.401120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a80b210d9794c48801202db6278e728)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:47.403334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:47.601063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a80b210d9794c48801202db6278e728'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:47.601756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a80b210d9794c48801202db6278e728'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:47.618472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a80b210d9794c48801202db6278e728'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:47.619501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a80b210d9794c48801202db6278e728'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:47.627948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a80b210d9794c48801202db6278e728'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:47.761926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1014/2075] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:47 2022] DELETE /identity/v3/users/6a80b210d9794c48801202db6278e728 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:18:47.766661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:47.767293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:47.767799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:47.791324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:47.793036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:47.875641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1062/2076] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:47.885728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:47.887452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:47.995085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69472-2c5f-435f-a420-528c2f1d2cc6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:47.999714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:48.000499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:48.000883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:48.001240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:48.013835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:48.015593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:48.027643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1015/2077] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:48.559289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:48.561672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:48.586451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bda16-4ead-41b5-988e-f13f7bd98c47 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:48.592300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:48.593493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:48.593841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:48.594350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] PATH_INFO: `/v3/projects/60b28373984a47cda134ce8849619220` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:48.595681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b28373984a47cda134ce8849619220)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:48.599112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:48.748916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b28373984a47cda134ce8849619220', 'request_id': 'req-17e2bd60-46db-4c46-aac9-f682150c8772'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:48.819180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b28373984a47cda134ce8849619220', 'request_id': 'req-17e2bd60-46db-4c46-aac9-f682150c8772'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:48.826049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b28373984a47cda134ce8849619220', 'request_id': 'req-17e2bd60-46db-4c46-aac9-f682150c8772'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:48.837861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] The token cache is being invalidate because project 60b28373984a47cda134ce8849619220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:48.838296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:48.839004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:48.840734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1063/2078] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:48 2022] DELETE /identity/v3/projects/60b28373984a47cda134ce8849619220 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:48.844785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:48.856112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b31b-9f7e-4a70-86a6-15b65c8dabdc service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:48.944262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:48.945311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:48.945606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:48.945941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] PATH_INFO: `/v3/users/ace26b9c782f4c709cdc2edb5fe2d846` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:48.947280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace26b9c782f4c709cdc2edb5fe2d846)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:48.949237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:49.176619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace26b9c782f4c709cdc2edb5fe2d846'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:49.177293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace26b9c782f4c709cdc2edb5fe2d846'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:49.192604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace26b9c782f4c709cdc2edb5fe2d846'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:49.194703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace26b9c782f4c709cdc2edb5fe2d846'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:49.204367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c8198007-5092-428b-9f51-b182f43e4b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace26b9c782f4c709cdc2edb5fe2d846'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:49.277526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1016/2079] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:18:48 2022] DELETE /identity/v3/users/ace26b9c782f4c709cdc2edb5fe2d846 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:49.808351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:49.809893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:49.846731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2bd60-46db-4c46-aac9-f682150c8772 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:49.914182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:49.915174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:49.915673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:49.916146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] PATH_INFO: `/v3/projects/0a636dce4e8c47488685088b7e855b80` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:49.917718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a636dce4e8c47488685088b7e855b80)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:49.920027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.042897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a636dce4e8c47488685088b7e855b80', 'request_id': 'req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:50.116253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a636dce4e8c47488685088b7e855b80', 'request_id': 'req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:50.133290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a636dce4e8c47488685088b7e855b80', 'request_id': 'req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:50.145853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] The token cache is being invalidate because project 0a636dce4e8c47488685088b7e855b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:18:50.146231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:50.146873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9a0aabbe-ee1d-4265-b1fe-599ca02f4612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:18:50.158292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1064/2080] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:18:49 2022] DELETE /identity/v3/projects/0a636dce4e8c47488685088b7e855b80 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:50.158528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.158528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.158528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.189643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:50.191064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:50.257025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1017/2081] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:50.265862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.266231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.266446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.285201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:50.286803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:50.307512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1065/2082] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:50.308430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.337511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079a534-7e15-4d95-b2e7-c53f78b13863 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.342316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.342989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.343271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.343545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.344275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:18:50.344574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.346161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.358245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1018/2083] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:18:50.362583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.372892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2629046-e667-4814-9df4-2c9d913ca688 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.377654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.378341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.379791 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.380070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.380762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.382739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.465749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1066/2084] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:50 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:50.469620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.480653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6280d-e692-4512-af47-3b34e7b7959b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.549210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.550217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.551114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.551517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.552241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.554392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.668347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1019/2085] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:50 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:50.669655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.682413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb54d-c5d3-4c76-befb-a48c99a7cc91 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.687666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.690320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.690320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.690320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.690320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.691382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.712675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1067/2086] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:50.716309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.732006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b151a63-4a4b-460b-946e-1a3580408519 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.736972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.737647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.737983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.738292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.739285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.740878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a6f7e-12c2-493f-b5a7-84de8c7c4c49 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.758780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1020/2087] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:18:50.763131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:50.776775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab955b-fa6d-4ca3-906e-063849b17f65 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:50.784066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:50.787025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.787477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.787987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] PATH_INFO: `/v3/projects/94cc69159fb64c27957c73bb4305f05f/users/8ac2a576746e4b6bbf319dca1527d757/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.816608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cc69159fb64c27957c73bb4305f05f, user_id=8ac2a576746e4b6bbf319dca1527d757, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:50.819685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:50.891618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1068/2088] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:50 2022] PUT /identity/v3/projects/94cc69159fb64c27957c73bb4305f05f/users/8ac2a576746e4b6bbf319dca1527d757/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:50.905665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:50.905995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:50.906263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:50.952369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] MFA Rules not processed for user `8ac2a576746e4b6bbf319dca1527d757`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:50.954172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:51.071308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1021/2089] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:51.072053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.141947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa25c6-6402-49a0-ab31-f413536816a4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.147861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.148532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.148816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.149091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.149782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.151806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.263423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1069/2090] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:18:51 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:18:51.267783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.286165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e5cea-e3e3-41f9-8692-d85276de659b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.365824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.366781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.367004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.367283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.368131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.370072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.445341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1022/2091] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:18:51 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:18:51.449657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.467413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf63b-b635-4e66-8bc1-f26c94ac571d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.474089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.474947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.475376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.475787 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.476668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.478447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.494991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1070/2092] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:51.499312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.511007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b14d-82d4-4dac-ac25-a200a8c40b7b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.516112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.517009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.517296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.517575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] PATH_INFO: `/v3/projects/f55c64f72c454c399d4912485a962502/users/36f775835c354975a651ca8151c9fa84/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.535732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55c64f72c454c399d4912485a962502, user_id=36f775835c354975a651ca8151c9fa84, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.539269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.598280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1023/2093] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:51 2022] PUT /identity/v3/projects/f55c64f72c454c399d4912485a962502/users/36f775835c354975a651ca8151c9fa84/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:51.600925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.618005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8390ba-d0f5-4973-bb62-13556bfa6cb9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.704040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.705317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.705689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.706057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.708221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.710603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88090b2f-24b9-4943-b718-1e2f7ec82c57 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.723187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1071/2094] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:18:51.727210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.739387 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5c38-0e00-4bd8-9075-df644dd4df8e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:51.744852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:51.746618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.746884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.747176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] PATH_INFO: `/v3/projects/f55c64f72c454c399d4912485a962502/users/36f775835c354975a651ca8151c9fa84/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.749343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55c64f72c454c399d4912485a962502, user_id=36f775835c354975a651ca8151c9fa84, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:51.752282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:51.764007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1024/2095] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:18:51 2022] PUT /identity/v3/projects/f55c64f72c454c399d4912485a962502/users/36f775835c354975a651ca8151c9fa84/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:18:51.775598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:51.775888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:51.776187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:51.808390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] MFA Rules not processed for user `36f775835c354975a651ca8151c9fa84`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:18:51.809750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:51.905901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1072/2096] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:18:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:18:51.914097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:51.915949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:18:52.005007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518788-b5af-4e6e-b98b-4e2b7a28e92e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:52.011436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:52.012551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:52.012844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:52.013330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:52.030140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:52.032040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:52.048308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1025/2097] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:18:52.292886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:18:52.305122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516cfa-1a39-4dea-90dc-3f80d75156a1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:18:52.309895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:18:52.310824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:18:52.311222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:18:52.311640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:18:52.377882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:18:52.379639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:18:52.400203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1073/2098] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:18:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:00.278183 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:00.280681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:00.381538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbf9b1-0517-4beb-ae65-df2b8ad24f23 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:00.389174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:00.390706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:00.391467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:00.392119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] PATH_INFO: `/v3/users/ef6b1765e6804fe784037b56fb7d6419` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:00.394031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6b1765e6804fe784037b56fb7d6419)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:00.398960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:00.588724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:00.604224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6b1765e6804fe784037b56fb7d6419'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.605790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6b1765e6804fe784037b56fb7d6419'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.607409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12457c-c34e-4fec-978b-6fbb244a697a service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:00.620579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:00.622053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:00.622053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:00.622591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] PATH_INFO: `/v3/users/8ac2a576746e4b6bbf319dca1527d757` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:00.623619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6b1765e6804fe784037b56fb7d6419'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.623780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac2a576746e4b6bbf319dca1527d757)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:00.624896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6b1765e6804fe784037b56fb7d6419'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.631197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:00.645700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca815aae-aeda-4473-b45d-9faa7c4c4a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6b1765e6804fe784037b56fb7d6419'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.815679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1026/2099] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:00 2022] DELETE /identity/v3/users/ef6b1765e6804fe784037b56fb7d6419 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:00.820165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:00.820609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:00.820904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:00.841845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:00.843222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:00.917238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1027/2100] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:00.929766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:00.932118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:00.944392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac2a576746e4b6bbf319dca1527d757'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.945408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac2a576746e4b6bbf319dca1527d757'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.959925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac2a576746e4b6bbf319dca1527d757'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.967580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac2a576746e4b6bbf319dca1527d757'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.968387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac2a576746e4b6bbf319dca1527d757'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:00.996480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a584-87d3-432f-a405-67ff8e608bb9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:01.000336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:01.001002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:01.001283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:01.001557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:01.003412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:01.017795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:01.019431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797c73b-9e2b-45b7-aa2c-e2a867ffc972 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:01.029319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1028/2101] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:19:01.083297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1074/2102] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:00 2022] DELETE /identity/v3/users/8ac2a576746e4b6bbf319dca1527d757 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:01.084856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:01.085875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:01.086649 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:01.108990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:01.110190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:01.169712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1029/2103] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:01.179778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:01.181194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:01.238765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c299-e428-40be-a91f-740fc2e9eb8b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:01.242432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:01.243146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:01.243350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:01.243626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:01.254661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:01.256400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:01.268417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1075/2104] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:01.679656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:01.681690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:01.699394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f1ae-0f20-41ab-aa41-67d97b66f30c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:01.713648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:01.716016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:01.716328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:01.716692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] PATH_INFO: `/v3/projects/64cdfef13989407893648db00239b75a` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:01.723090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64cdfef13989407893648db00239b75a)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:01.728109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:01.859741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64cdfef13989407893648db00239b75a', 'request_id': 'req-ca14fe3a-6649-4167-b1b7-08c881805dc5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:01.934902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64cdfef13989407893648db00239b75a', 'request_id': 'req-ca14fe3a-6649-4167-b1b7-08c881805dc5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:01.949021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:01.950927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:01.964117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64cdfef13989407893648db00239b75a', 'request_id': 'req-ca14fe3a-6649-4167-b1b7-08c881805dc5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:01.989121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be388b06-06ec-4cfe-9cb8-8574ba7003e2 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:01.991704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] The token cache is being invalidate because project 64cdfef13989407893648db00239b75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:01.992783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:01.993668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:01.999028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1030/2105] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:01 2022] DELETE /identity/v3/projects/64cdfef13989407893648db00239b75a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:02.089205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:02.090274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:02.090621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:02.090902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] PATH_INFO: `/v3/projects/94cc69159fb64c27957c73bb4305f05f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:02.092235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cc69159fb64c27957c73bb4305f05f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:02.094425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:02.265091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cc69159fb64c27957c73bb4305f05f', 'request_id': 'req-96596b74-6705-411a-85c9-6fac60fcbcf3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.350581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cc69159fb64c27957c73bb4305f05f', 'request_id': 'req-96596b74-6705-411a-85c9-6fac60fcbcf3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.360937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cc69159fb64c27957c73bb4305f05f', 'request_id': 'req-96596b74-6705-411a-85c9-6fac60fcbcf3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.372776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] The token cache is being invalidate because project 94cc69159fb64c27957c73bb4305f05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:02.373114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.373826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.379663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:02.380173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1076/2106] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:01 2022] DELETE /identity/v3/projects/94cc69159fb64c27957c73bb4305f05f => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:02.381316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:02.396110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14fe3a-6649-4167-b1b7-08c881805dc5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:02.477977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:02.478982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:02.479333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:02.479686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] PATH_INFO: `/v3/users/36f775835c354975a651ca8151c9fa84` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:02.481355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36f775835c354975a651ca8151c9fa84)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:02.483633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:02.685977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36f775835c354975a651ca8151c9fa84'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.687036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36f775835c354975a651ca8151c9fa84'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.700711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36f775835c354975a651ca8151c9fa84'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.701453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36f775835c354975a651ca8151c9fa84'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.709330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0e4be9c9-c219-417d-84dd-b87ad7d33576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36f775835c354975a651ca8151c9fa84'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:02.788233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1031/2107] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:02 2022] DELETE /identity/v3/users/36f775835c354975a651ca8151c9fa84 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:03.292072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:03.294151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:03.321228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596b74-6705-411a-85c9-6fac60fcbcf3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:03.401374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:03.402273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:03.402562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:03.402861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] PATH_INFO: `/v3/projects/f55c64f72c454c399d4912485a962502` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:03.404093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f55c64f72c454c399d4912485a962502)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:03.406058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:03.557076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f55c64f72c454c399d4912485a962502', 'request_id': 'req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:03.624259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f55c64f72c454c399d4912485a962502', 'request_id': 'req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:03.631141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f55c64f72c454c399d4912485a962502', 'request_id': 'req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:03.642917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] The token cache is being invalidate because project f55c64f72c454c399d4912485a962502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:03.643541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:03.644468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-de884b96-ebf4-4e3f-95b4-ad7c4484cfb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:03.646954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1077/2108] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:03 2022] DELETE /identity/v3/projects/f55c64f72c454c399d4912485a962502 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:03.656718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:03.662821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:03.662821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:03.680277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:03.682098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:03.774128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1032/2109] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:03.776857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:03.777366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:03.777815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:03.798082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:03.799539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:03.809531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1078/2110] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:03.813816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:03.825936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13491-5e8a-4145-873e-d733b568d0a3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:03.831159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:03.832523 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:03.832808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:03.833085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:03.833894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:19:03.834241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:03.835926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:03.849630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1033/2111] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:19:03.861973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:03.873624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dce617-6778-4b86-94a8-875dcc600327 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:03.878934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:03.879816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:03.880101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:03.880368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:03.881070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:03.883069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.029490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1079/2112] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:03 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:04.030912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:04.045337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd5271-1d15-4373-8126-f663acd397aa admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:04.119590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:04.120351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.120763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.121173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.121967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:04.124023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.229498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1034/2113] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:04 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:04.234120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:04.248287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230a6c9-e7be-4bd3-a553-773062a07b88 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:04.253312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:04.254043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.254366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.254646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.255536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:04.257398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.268749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1080/2114] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:04.272998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:04.284803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb5abc-eb56-4f8b-9983-e6cb7918623e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:04.291507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:04.292321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.292509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.292775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.293593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:04.295515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e0d17-6866-4be5-bbcf-6732ee97204d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.307762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1035/2115] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:04.311847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:04.329296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e22a0-6b8c-46bd-8cf7-7c17e3dfb2f1 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:04.336218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:04.337426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.337800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.338166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] PATH_INFO: `/v3/projects/23fef991e30d4ff7b0fec1040239d9fb/users/8dbffa6b71bd4e8cb7d2e744127a279b/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.367948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fef991e30d4ff7b0fec1040239d9fb, user_id=8dbffa6b71bd4e8cb7d2e744127a279b, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:04.371415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.448385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1081/2116] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:04 2022] PUT /identity/v3/projects/23fef991e30d4ff7b0fec1040239d9fb/users/8dbffa6b71bd4e8cb7d2e744127a279b/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:04.462242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5532c35-7c6a-461e-87f6-bd4c8fe31c54 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.462242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5532c35-7c6a-461e-87f6-bd4c8fe31c54 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.462242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5532c35-7c6a-461e-87f6-bd4c8fe31c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.505686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d5532c35-7c6a-461e-87f6-bd4c8fe31c54 None None] MFA Rules not processed for user `8dbffa6b71bd4e8cb7d2e744127a279b`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:04.507151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d5532c35-7c6a-461e-87f6-bd4c8fe31c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:04.593170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1036/2117] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:04.605788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:04.607537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:04.666004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494e437-f4c8-40c1-83bb-3a31a582dbbc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:04.670187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:04.670912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:04.671202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:04.671473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:04.681313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:04.682958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac4df2-0808-4c5b-b318-1cd0b8b0de7c service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:04.694251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1082/2118] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:07.144549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:07.145326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:07.145610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:07.164347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] MFA Rules not processed for user `5e3edc87e7014d8598ba8c9517b81cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:07.165651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:07.218282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1037/2119] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:07.812937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:07.813445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:07.813540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:07.863040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:07.864624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:07.938398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1083/2120] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 10 14:19:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:07.955151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:07.957742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:08.093056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0dc8e-f076-4c00-b2e5-2a102eb57d26 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:08.098362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:08.099021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:08.099306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:08.099588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:08.108395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:08.109966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694caa3-a53c-4b4a-81da-888137e03475 service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:08.118856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1038/2121] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:19:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:19:09.993916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:10.061588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5f96-1c8a-4306-bd59-093b858943d7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:10.066389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:10.067458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:10.067736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:10.068010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] PATH_INFO: `/v3/users/8dbffa6b71bd4e8cb7d2e744127a279b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:10.069305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dbffa6b71bd4e8cb7d2e744127a279b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:10.071279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:10.249715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dbffa6b71bd4e8cb7d2e744127a279b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:10.250348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dbffa6b71bd4e8cb7d2e744127a279b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:10.262567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dbffa6b71bd4e8cb7d2e744127a279b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:10.269288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dbffa6b71bd4e8cb7d2e744127a279b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:10.270733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dbffa6b71bd4e8cb7d2e744127a279b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:10.333482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1084/2122] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:09 2022] DELETE /identity/v3/users/8dbffa6b71bd4e8cb7d2e744127a279b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:10.338107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:10.338625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:10.338716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:10.356356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:10.357557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:10.417773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1039/2123] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:10.428244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:10.429771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:10.491447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb8f03-8fe7-4cff-8c46-bc55802b0d87 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:10.495419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:10.496075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:10.496361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:10.496635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:10.506980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:10.508127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:10.518332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1085/2124] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:11.531972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:11.533393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:11.545509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fa8a7-5471-4a0a-97ed-c574e2942662 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:11.550177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:11.551833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:11.552112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:11.552393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] PATH_INFO: `/v3/projects/23fef991e30d4ff7b0fec1040239d9fb` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:11.553920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=23fef991e30d4ff7b0fec1040239d9fb)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:11.558127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:11.660466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23fef991e30d4ff7b0fec1040239d9fb', 'request_id': 'req-11a57e52-aea2-4e7b-8602-547f6541fa9c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:11.787438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:11.789477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:11.802080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23fef991e30d4ff7b0fec1040239d9fb', 'request_id': 'req-11a57e52-aea2-4e7b-8602-547f6541fa9c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:11.804237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a151-2b28-4010-b01d-d68f974bb942 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:11.808339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23fef991e30d4ff7b0fec1040239d9fb', 'request_id': 'req-11a57e52-aea2-4e7b-8602-547f6541fa9c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:11.820688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] The token cache is being invalidate because project 23fef991e30d4ff7b0fec1040239d9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:11.820989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:11.821638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-11a57e52-aea2-4e7b-8602-547f6541fa9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:11.823242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1040/2125] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:11 2022] DELETE /identity/v3/projects/23fef991e30d4ff7b0fec1040239d9fb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:11.828288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee30be-f71c-4cc8-a38e-a59cafc7914a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:11.828556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee30be-f71c-4cc8-a38e-a59cafc7914a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:11.828871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee30be-f71c-4cc8-a38e-a59cafc7914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:11.848248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-69ee30be-f71c-4cc8-a38e-a59cafc7914a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:11.849752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-69ee30be-f71c-4cc8-a38e-a59cafc7914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:11.861467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:11.862357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:11.862639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:11.862996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] PATH_INFO: `/v3/users/7669f2f83550428893d6c91614b78f76` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:11.864998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7669f2f83550428893d6c91614b78f76)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:11.867114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:11.915999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1041/2126] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:11.924004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:11.924508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:11.924964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:11.945855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:11.945855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:11.957902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1042/2127] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:11.959106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:11.968943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef607fc-6a12-4b52-bafe-b713e1aa9f87 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:11.974039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:11.974802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:11.975082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:11.975367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:11.976122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:19:11.976427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:11.978111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:11.991298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1043/2128] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:19:11.996351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.007961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a5ac3-b4f2-4f6b-bc8e-5f2708db7aa6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.014163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.014969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.015252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.015531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.016244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.018989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.056599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7669f2f83550428893d6c91614b78f76'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.057322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7669f2f83550428893d6c91614b78f76'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.070698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7669f2f83550428893d6c91614b78f76'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.078065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7669f2f83550428893d6c91614b78f76'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.078826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7669f2f83550428893d6c91614b78f76'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.112115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1044/2129] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:11 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:12.116332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.118055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:12.167383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1086/2130] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:11 2022] DELETE /identity/v3/users/7669f2f83550428893d6c91614b78f76 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:12.189537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883ccae-0994-48d6-af6a-d475bca99547 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.251681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.252467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.260272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.260272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.260272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.260272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.349528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1045/2131] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:12.353482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.365360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f3ae-7ad1-4cf8-8d4b-7e43f0c5848b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.377413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.377413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.377413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.377413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.377746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.378018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.389504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1087/2132] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:12.393972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.408502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd4ecb-9e8b-4d86-92dd-45af289bb4c0 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.416330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.417198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.417505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.417850 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.418630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.420397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.435852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1046/2133] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:12.439940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.463490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5af5-6725-4bad-a233-42662f7a2f50 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.470941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.472127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.472478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.473054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] PATH_INFO: `/v3/projects/66f0da87b76241d0a732d78f0545f48e/users/d65375387239471da2b59a09bff8bdc4/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.490379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f0da87b76241d0a732d78f0545f48e, user_id=d65375387239471da2b59a09bff8bdc4, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.494593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac6bc0-ec1e-41bd-a47a-27fac1e7aeee admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.535090 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.537293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:12.553043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f383-efec-443a-96fb-c67debf820c9 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.559567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1088/2134] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:12 2022] PUT /identity/v3/projects/66f0da87b76241d0a732d78f0545f48e/users/d65375387239471da2b59a09bff8bdc4/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:12.570932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.571235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.571949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.602159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] MFA Rules not processed for user `d65375387239471da2b59a09bff8bdc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:12.603602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:12.657824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.658739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.659018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.659295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] PATH_INFO: `/v3/projects/2c1037d775114df4b21dbdfa5f639cb6` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.660694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1037d775114df4b21dbdfa5f639cb6)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.662646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.678162 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1089/2135] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:12.682655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.693722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b80b78-b8e6-45fc-a020-1bf11597146a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.698357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.699013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.699293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.699565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.700250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.702354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.836872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1037d775114df4b21dbdfa5f639cb6', 'request_id': 'req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.877514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1090/2136] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:12.881667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:12.911576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1037d775114df4b21dbdfa5f639cb6', 'request_id': 'req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.919157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1037d775114df4b21dbdfa5f639cb6', 'request_id': 'req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.933159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] The token cache is being invalidate because project 2c1037d775114df4b21dbdfa5f639cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:12.933489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.934142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e6c081ae-ec3b-41ac-8ae7-6a42de453da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:12.935784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1047/2137] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:12 2022] DELETE /identity/v3/projects/2c1037d775114df4b21dbdfa5f639cb6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:12.942613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbe219-d2f3-40f0-b4b2-c2a710497b76 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.942904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbe219-d2f3-40f0-b4b2-c2a710497b76 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.943185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbe219-d2f3-40f0-b4b2-c2a710497b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.947559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5594e-5317-47f5-996b-8c98cec42582 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:12.952266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:12.952946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.953237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.953519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:12.954276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:12.956216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:12.962868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4fdbe219-d2f3-40f0-b4b2-c2a710497b76 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:12.964077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4fdbe219-d2f3-40f0-b4b2-c2a710497b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:12.972330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1048/2138] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:12.986173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:12.986490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:12.986770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.005743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:13.007272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:13.015667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1049/2139] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:13.022170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.027440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1091/2140] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:12 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:13.031700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.034722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:13.036138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb2a2-470c-4469-9266-d5b73bd9c98e None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.045574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.046737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.047246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.047746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.049162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:19:13.049715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.052897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.059862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61ae4f-7a7b-4ae3-9853-616ade504774 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.065680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.066096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.067494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.067494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.068095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.069310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1050/2141] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:19:13.069898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.073682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.082802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1092/2142] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:13.085498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec251323-e24f-4504-88d5-752408a6850d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.085767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.090218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.090830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.091113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.091385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.092071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.093969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.096142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6d905-c600-47b9-aa5e-929c406159f3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.101986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.102674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.102957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.103224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.103945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.105512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.117452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1093/2143] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:13.121474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.131848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5f827-bd5e-4701-808f-4f40afce5e94 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.136722 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.137575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.137858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.138133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] PATH_INFO: `/v3/projects/b92d79d1ed3c4713b7c09d1324a56de9/users/d6f901c932d84cd181a296588fbcddc8/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.153064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b92d79d1ed3c4713b7c09d1324a56de9, user_id=d6f901c932d84cd181a296588fbcddc8, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.155775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b199a4-31c9-4eb6-85b0-5acab175876b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.191912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1051/2144] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:13.196113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.256316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89703fff-512b-446a-88a8-7e25b4a48541 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.259456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1094/2145] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:13 2022] PUT /identity/v3/projects/b92d79d1ed3c4713b7c09d1324a56de9/users/d6f901c932d84cd181a296588fbcddc8/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:13.270432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.270850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.271045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.297494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] MFA Rules not processed for user `d6f901c932d84cd181a296588fbcddc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:13.298772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:13.313315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.313991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.314308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.314587 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.315284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.317733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.365247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1095/2146] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:13.369749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.380238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460a949-b43d-4a85-a203-d9d63d6160d4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.385133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.385781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.386061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.388675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.388675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.389287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.393704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1052/2147] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:13.397325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.407294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2c75-5fb1-41ed-a5f0-030ab94f7e73 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.413234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.413959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.414277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.414554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.417298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.417298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.428588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1053/2148] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:13.434789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.444916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0873d-cce2-45ff-892e-660da12be2d8 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.449464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.450108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.450426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.450704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.451412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.452992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.461500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1096/2149] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:19:13.464599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1054/2150] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:13.465658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.471761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.481430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23517-505b-4472-acf2-753ea4146d42 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.525829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020c8e8-a070-4392-aff2-b11fd8ed2f45 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.530561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.531207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.531492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.531769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.532485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.534417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.535471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.536520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.536795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.541136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] PATH_INFO: `/v3/projects/a1277ec56a7c4b199e9f7e04d023b1f1/users/fee076042f7647829cf676d7d9f882d3/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.552321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1277ec56a7c4b199e9f7e04d023b1f1, user_id=fee076042f7647829cf676d7d9f882d3, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.554999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b163e52-a921-42bc-83f8-d1cabd1f54aa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.619644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1097/2151] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:13.623634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.634389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382fc4f-952e-4d94-b4ed-f380a669a587 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.639057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.639712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.640155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.640494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.641188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.642986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.650070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1055/2152] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:13 2022] PUT /identity/v3/projects/a1277ec56a7c4b199e9f7e04d023b1f1/users/fee076042f7647829cf676d7d9f882d3/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:13.659935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1098/2153] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:13.660176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.660661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.661162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.664355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.675217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34601c-9570-414f-b36b-f93a3b0ef59b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.689412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] MFA Rules not processed for user `fee076042f7647829cf676d7d9f882d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:13.690639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:13.740163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.741456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.741740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.742358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] PATH_INFO: `/v3/projects/6676ec77e6ba4ab29c1510492bb13c48/users/d2dafcc807bc4950963acf8b8a4b8383/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.762167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6676ec77e6ba4ab29c1510492bb13c48, user_id=d2dafcc807bc4950963acf8b8a4b8383, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.765928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.769927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1056/2154] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:13.781719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.783287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:13.841600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270cb27-72a0-4718-b63d-5e2b5b0f9d4d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.848268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1099/2155] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:13 2022] PUT /identity/v3/projects/6676ec77e6ba4ab29c1510492bb13c48/users/d2dafcc807bc4950963acf8b8a4b8383/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:13.851840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.892422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.893078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.893369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.893647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.914428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29850a72-eb81-486b-9197-4eeaa866902a admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.919618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.920289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.920578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.920860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.921622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.923263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.934008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1100/2156] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:13.937895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:13.947393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef914d9-8c76-4827-bed5-25b463f51181 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:13.952048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:13.952911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.953046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.953195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.953479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] PATH_INFO: `/v3/projects/6676ec77e6ba4ab29c1510492bb13c48/users/d2dafcc807bc4950963acf8b8a4b8383/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:13.954521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81668c10-f509-4189-8502-4ad41d25a7e8 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.955483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6676ec77e6ba4ab29c1510492bb13c48, user_id=d2dafcc807bc4950963acf8b8a4b8383, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:13.958490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:13.963725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1057/2157] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:19:13.968690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1101/2158] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:13 2022] PUT /identity/v3/projects/6676ec77e6ba4ab29c1510492bb13c48/users/d2dafcc807bc4950963acf8b8a4b8383/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:13.979301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:13.979591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:13.979870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:14.006535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] MFA Rules not processed for user `d2dafcc807bc4950963acf8b8a4b8383`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:14.007737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:14.075538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1058/2159] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:14.086970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:14.088746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:14.145876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b7869-b5e0-451d-8d27-e4fc87f637e4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:14.149629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:14.150296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:14.150583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:14.150878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:14.152664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:14.211697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:14.213139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:14.222006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1102/2160] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:17.250801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:17.260863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c416d2-d383-4681-bb1d-664c406e2033 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:17.264427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:17.265069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:17.265354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:17.265629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:17.325048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:17.326625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70effa-f4b1-4cf0-ae3c-8817ed88404b service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:17.336959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1059/2161] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:18.386136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:18.460945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d57a-8d53-4ea7-a14b-b483530c56f4 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:18.466661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:18.467748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:18.468030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:18.468309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] PATH_INFO: `/v3/users/fee076042f7647829cf676d7d9f882d3` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:18.469802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee076042f7647829cf676d7d9f882d3)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:18.471755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:18.604113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee076042f7647829cf676d7d9f882d3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:18.604776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee076042f7647829cf676d7d9f882d3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:18.617057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee076042f7647829cf676d7d9f882d3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:18.623593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee076042f7647829cf676d7d9f882d3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:18.624269 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee076042f7647829cf676d7d9f882d3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:18.722326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1103/2162] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:18 2022] DELETE /identity/v3/users/fee076042f7647829cf676d7d9f882d3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:18.760758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:18.760758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:18.760758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:18.777145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:18.778415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:18.835703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1060/2163] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:18.845648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:18.847096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:18.903688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9303c6-e7cf-4864-bfcb-551005cd3fbd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:18.907408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:18.908077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:18.908356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:18.908629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:18.918143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:18.919601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:18.929579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1104/2164] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:19.074528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:19.075942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:19.141161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e667a4-ee0b-4b86-8f9d-f1aa8ccf069a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:19.149188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:19.149863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.150142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.150448 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.152398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:19.241905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:19.243923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75fdcb-4fb8-4fed-9978-f4e743d0c81c service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:19.254263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1061/2165] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:19.399402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:19.400890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:19.415544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd22626-4283-4158-9321-d07b1bc0f06a service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:19.420264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:19.421116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.421395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.421665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] PATH_INFO: `/v3/projects/a1277ec56a7c4b199e9f7e04d023b1f1` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.422869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1277ec56a7c4b199e9f7e04d023b1f1)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:19.424776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:19.542326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1277ec56a7c4b199e9f7e04d023b1f1', 'request_id': 'req-083b523d-f8d1-46ce-8a92-cd532c32f2a7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:19.608515 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1277ec56a7c4b199e9f7e04d023b1f1', 'request_id': 'req-083b523d-f8d1-46ce-8a92-cd532c32f2a7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:19.616147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1277ec56a7c4b199e9f7e04d023b1f1', 'request_id': 'req-083b523d-f8d1-46ce-8a92-cd532c32f2a7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:19.630016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] The token cache is being invalidate because project a1277ec56a7c4b199e9f7e04d023b1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:19.630690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:19.631409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-083b523d-f8d1-46ce-8a92-cd532c32f2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:19.633241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1105/2166] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:19 2022] DELETE /identity/v3/projects/a1277ec56a7c4b199e9f7e04d023b1f1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:19.639267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.640058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.640552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.661960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:19.663497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:19.726856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1062/2167] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:19.734867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.735190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.735469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.756271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:19.757463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:19.765772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1106/2168] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:19.773864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:19.783452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabbeaa-5fc3-4c89-bcee-ebc9ba1e8587 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:19.789035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:19.789686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.789970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.790284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.791020 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:19:19.791328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:19.792923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:19.804837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1063/2169] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:19:19.809084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:19.818230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68d63a-8075-4e46-b778-bde0234b7caa None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:19.822957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:19.823596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:19.825767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:19.825767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:19.825767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:19.826797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:19.952376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1107/2170] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:19 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:19.956419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:19.966575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8ab4-c59f-4e4e-9f3c-d56da4304262 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:20.021922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:20.022610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.022892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.023171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.023855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:20.025746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:20.117108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1064/2171] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:19 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:20.121240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:20.131479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377c8f-7a14-4399-90be-34567c65a65d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:20.136086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:20.137565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.137565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.137565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.138054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:20.139674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:20.151033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1108/2172] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:20.155053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:20.166369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d5c13-14eb-4bf6-b056-4bd3eafbbf59 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:20.171193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:20.171841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.172123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.172394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.173112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:20.174725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f484b-bc58-4887-9008-d72620fe5f6f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:20.184714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1065/2173] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:19:20.188701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:20.198850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbf24f-6136-40ac-96a4-fd44ce759595 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:20.203453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:20.204370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.204667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.204941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] PATH_INFO: `/v3/projects/ac430abda2c64e3fad2daec30d41778e/users/6e18569dc0f84706a7b4d6d68ede47e3/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.219660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac430abda2c64e3fad2daec30d41778e, user_id=6e18569dc0f84706a7b4d6d68ede47e3, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:20.222337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:20.339533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1109/2174] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:20 2022] PUT /identity/v3/projects/ac430abda2c64e3fad2daec30d41778e/users/6e18569dc0f84706a7b4d6d68ede47e3/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:20.357491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.357786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.358069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.383886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] MFA Rules not processed for user `6e18569dc0f84706a7b4d6d68ede47e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:20.385104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:20.447317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1066/2175] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:20.459111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:20.460776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:20.534718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71766c-3912-4b56-bd2c-54f1bce18384 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:20.538725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:20.539368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:20.539647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:20.539920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:20.548330 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:20.549746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:20.558852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1110/2176] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:40.074604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:40.076012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:40.163364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e043d52-b289-4d1e-82ba-04c9ead4450b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:40.169032 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:40.169682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:40.169963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:40.170261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:40.172024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:40.183760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:40.185331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0c28c-2fa9-4753-87aa-e934197b650b service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:40.195654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1067/2177] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:41.809965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:41.811428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:41.877436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494b80f-a98b-4a02-9678-6f34f5794b67 service glance] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:41.883220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:41.884088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:41.884371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:41.884675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] PATH_INFO: `/v3/users/d65375387239471da2b59a09bff8bdc4` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:41.885861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65375387239471da2b59a09bff8bdc4)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:41.887873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:42.084675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65375387239471da2b59a09bff8bdc4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:42.085357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65375387239471da2b59a09bff8bdc4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:42.096717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65375387239471da2b59a09bff8bdc4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:42.103685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65375387239471da2b59a09bff8bdc4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:42.104395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65375387239471da2b59a09bff8bdc4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:42.168991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1111/2178] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:41 2022] DELETE /identity/v3/users/d65375387239471da2b59a09bff8bdc4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:42.173648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:42.174175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:42.174300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:42.194187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:42.195410 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:42.255257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1068/2179] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:42.265760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:42.267281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:42.331200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147f884-3470-4739-b075-c9034edcd1e5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:42.335687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:42.339197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:42.339197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:42.339197 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:42.345849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:42.347311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:42.356919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1112/2180] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:42.781575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:42.783029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:42.795656 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845853f-31ce-44cf-a8af-92b146f87643 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:42.800255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:42.801142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:42.801929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:42.802452 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] PATH_INFO: `/v3/projects/66f0da87b76241d0a732d78f0545f48e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:42.803635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f0da87b76241d0a732d78f0545f48e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:42.805571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:42.929619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f0da87b76241d0a732d78f0545f48e', 'request_id': 'req-75189103-081c-443a-92cc-d6691c56b29d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.003009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f0da87b76241d0a732d78f0545f48e', 'request_id': 'req-75189103-081c-443a-92cc-d6691c56b29d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.009643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f0da87b76241d0a732d78f0545f48e', 'request_id': 'req-75189103-081c-443a-92cc-d6691c56b29d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.021525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] The token cache is being invalidate because project 66f0da87b76241d0a732d78f0545f48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:43.021915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.022795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.024699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1069/2181] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:42 2022] DELETE /identity/v3/projects/66f0da87b76241d0a732d78f0545f48e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:43.028799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:43.039190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbaf70b-90d1-4ba2-ad37-fe30bfbf94b4 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:43.092614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:43.093527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:43.093815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:43.094082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] PATH_INFO: `/v3/users/d6f901c932d84cd181a296588fbcddc8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:43.095323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f901c932d84cd181a296588fbcddc8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:43.097253 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:43.302608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f901c932d84cd181a296588fbcddc8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.303263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f901c932d84cd181a296588fbcddc8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.315126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f901c932d84cd181a296588fbcddc8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.325901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f901c932d84cd181a296588fbcddc8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.327068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f901c932d84cd181a296588fbcddc8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:43.380158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1113/2182] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:43 2022] DELETE /identity/v3/users/d6f901c932d84cd181a296588fbcddc8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:19:43.793926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:43.799231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:43.811089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75189103-081c-443a-92cc-d6691c56b29d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:43.867123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:43.867981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:43.868262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:43.868535 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] PATH_INFO: `/v3/projects/b92d79d1ed3c4713b7c09d1324a56de9` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:43.869704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b92d79d1ed3c4713b7c09d1324a56de9)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:43.871652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:43.974222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b92d79d1ed3c4713b7c09d1324a56de9', 'request_id': 'req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.043016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b92d79d1ed3c4713b7c09d1324a56de9', 'request_id': 'req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.049068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b92d79d1ed3c4713b7c09d1324a56de9', 'request_id': 'req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.060069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] The token cache is being invalidate because project b92d79d1ed3c4713b7c09d1324a56de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:44.060366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.061027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.062611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1070/2183] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:43 2022] DELETE /identity/v3/projects/b92d79d1ed3c4713b7c09d1324a56de9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:44.066543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:44.068490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:44.081273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655752d6-03d2-4d49-930b-446c72b0ddac admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:44.141126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:44.142020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:44.142337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:44.142617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] PATH_INFO: `/v3/users/d2dafcc807bc4950963acf8b8a4b8383` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:44.143814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2dafcc807bc4950963acf8b8a4b8383)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:44.146117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:44.344481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2dafcc807bc4950963acf8b8a4b8383'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.346058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2dafcc807bc4950963acf8b8a4b8383'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.359815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2dafcc807bc4950963acf8b8a4b8383'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.365858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2dafcc807bc4950963acf8b8a4b8383'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.366598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e4220756-11e9-40d5-8068-fe728aa96506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2dafcc807bc4950963acf8b8a4b8383'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:44.442806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1114/2184] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:19:44 2022] DELETE /identity/v3/users/d2dafcc807bc4950963acf8b8a4b8383 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:44.885401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:44.887435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:44.904949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3e5b3-1396-40e9-bf76-d5d3b4d29752 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:44.969271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:44.970132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:44.970445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:44.970722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] PATH_INFO: `/v3/projects/6676ec77e6ba4ab29c1510492bb13c48` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:44.971894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6676ec77e6ba4ab29c1510492bb13c48)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:44.974491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.095907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6676ec77e6ba4ab29c1510492bb13c48', 'request_id': 'req-c94d17fc-1a5e-4610-badd-6dbdb031c717'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:45.146548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6676ec77e6ba4ab29c1510492bb13c48', 'request_id': 'req-c94d17fc-1a5e-4610-badd-6dbdb031c717'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:45.152573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6676ec77e6ba4ab29c1510492bb13c48', 'request_id': 'req-c94d17fc-1a5e-4610-badd-6dbdb031c717'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:45.165110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] The token cache is being invalidate because project 6676ec77e6ba4ab29c1510492bb13c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:19:45.165420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:45.166083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-c94d17fc-1a5e-4610-badd-6dbdb031c717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:19:45.167705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1071/2185] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:19:44 2022] DELETE /identity/v3/projects/6676ec77e6ba4ab29c1510492bb13c48 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:45.173141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.174030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.174504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.193350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:45.194739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:45.255227 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1115/2186] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:19:45.263420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.263711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.264047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.286133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:45.288099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:45.297064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1072/2187] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:45.303971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.315316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f8f75-b0cb-4a9f-b557-12aa98005f4b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.324233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.325457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.325744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.326815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.327511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:19:45.327820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.329551 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.344294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1116/2188] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:19:45.349265 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.363677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c250f-15f8-48e1-8205-9f5adea81f1f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.371923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.372892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.373416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.374015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.374843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.377008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.491378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1073/2189] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:45 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:45.495460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.505438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ea876-344b-49cd-a8dd-bd24f635d489 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.560141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.560821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.561104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.561379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.562073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.564062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.675578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1117/2190] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:45 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:45.676908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.688813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8d304-3a50-4042-ba6c-ea51c9ebe1bf admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.693810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.694597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.694946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.695320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.696121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.697712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.709054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1074/2191] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:45.713268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.724463 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6b3c-09f5-47cb-b1e5-fb727bdb56bc admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.729283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.729945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.730275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.730541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.731266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.732862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6ff41-4605-4939-98ab-af83bb9ba0c4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.742779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1118/2192] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:45.747057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.756997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1df44-27e2-4f35-bcf9-d6c0c2d3e643 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:45.761553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:45.762459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.762741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.763016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] PATH_INFO: `/v3/projects/674aa2ca44674c908e6393ef25d6719a/users/98be3f9cd80e4f62972dbf48591c7dd4/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.777576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=674aa2ca44674c908e6393ef25d6719a, user_id=98be3f9cd80e4f62972dbf48591c7dd4, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:45.780313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:45.877102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1075/2193] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:45 2022] PUT /identity/v3/projects/674aa2ca44674c908e6393ef25d6719a/users/98be3f9cd80e4f62972dbf48591c7dd4/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:45.882296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec3bb-0bdd-4a27-a135-6e309e6c76ae None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:45.882589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec3bb-0bdd-4a27-a135-6e309e6c76ae None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:45.882874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec3bb-0bdd-4a27-a135-6e309e6c76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:45.901143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-32bec3bb-0bdd-4a27-a135-6e309e6c76ae None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:45.902353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-32bec3bb-0bdd-4a27-a135-6e309e6c76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:45.967472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1119/2194] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:45.974700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:45.976408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:46.037794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc91a-4045-4f96-9cca-2fb07b54f2ef admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:46.041808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:46.042500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:46.042784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:46.043062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:46.053031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:46.054527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:46.064358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1076/2195] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:19:52.210762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.211362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.211362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.240529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] MFA Rules not processed for user `98be3f9cd80e4f62972dbf48591c7dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:52.241936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:52.307580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1120/2196] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:52.312811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.325051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fde6c-0f58-4bba-b48a-290163870291 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.329976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.330698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.331176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.331620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.333412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.335900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.434616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1077/2197] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:19:52 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:52.439161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.450766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938495df-c8ee-4ffc-9f3c-259eef1dc6ae None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.507950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.508659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.508987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.509270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.509956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.512040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.604695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1121/2198] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:19:52 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:19:52.608848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.620329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ff7-c76b-4eb6-b6e8-8519385c3fde admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.625383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.626048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.626371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.626647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.627374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.629023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.641065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1078/2199] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:52.646069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.656567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7464e7-b9e6-4f64-b692-cda58b850cc9 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.662797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.663808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.664093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.664381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] PATH_INFO: `/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5/users/5230438fb71d417bb5142bd09b160607/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.681482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228ca61d6fec4e0397cdfc00f0126ed5, user_id=5230438fb71d417bb5142bd09b160607, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.684374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.764349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1122/2200] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:52 2022] PUT /identity/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5/users/5230438fb71d417bb5142bd09b160607/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:52.768497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.784654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314714f-3b5a-49d9-955e-2477e579c5cf admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.852046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.852734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.853017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.853333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.854018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.856530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8529c6-1605-4c51-9975-3187313080b2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.871581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1079/2201] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:19:52.872301 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:52.883210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7859-e3a3-4144-bc04-bfa857254157 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:52.888132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:52.889017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:52.889296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:52.889586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] PATH_INFO: `/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5/users/5230438fb71d417bb5142bd09b160607/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:52.891686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] RBAC: Authorizing `identity:create_grant(project_id=228ca61d6fec4e0397cdfc00f0126ed5, user_id=5230438fb71d417bb5142bd09b160607, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:52.894354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:52.905557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1123/2202] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:19:52 2022] PUT /identity/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5/users/5230438fb71d417bb5142bd09b160607/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:19:58.769606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde71e9-8dbe-483d-a855-5804b616f0cb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:58.770084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde71e9-8dbe-483d-a855-5804b616f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:58.770084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde71e9-8dbe-483d-a855-5804b616f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:58.813829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0dde71e9-8dbe-483d-a855-5804b616f0cb None None] MFA Rules not processed for user `5230438fb71d417bb5142bd09b160607`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:19:58.815373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0dde71e9-8dbe-483d-a855-5804b616f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:58.887517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1080/2203] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:19:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:19:58.902073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:19:58.904095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:19:59.004006 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d4e1b-8818-493d-88e5-ee1dec05223e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:19:59.010577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:19:59.011231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:19:59.011514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:19:59.011792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:19:59.070142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:19:59.071829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:19:59.080952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1124/2204] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:19:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:20:07.118842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6365-d7c0-4a3c-80ca-cdbcc6f5bcfe None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:07.119382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6365-d7c0-4a3c-80ca-cdbcc6f5bcfe None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:07.119382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6365-d7c0-4a3c-80ca-cdbcc6f5bcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:07.137158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4f1a6365-d7c0-4a3c-80ca-cdbcc6f5bcfe None None] MFA Rules not processed for user `48efa16cb9844fcd853f1e3760619a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:07.138400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4f1a6365-d7c0-4a3c-80ca-cdbcc6f5bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:07.196109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1081/2205] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:25.515379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:25.516815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:25.580193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae9b4-4ab5-4c9c-ab9a-cd44c23df0ff service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:25.585201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:25.586056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:25.586396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:25.586675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] PATH_INFO: `/v3/users/6e18569dc0f84706a7b4d6d68ede47e3` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:25.587852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e18569dc0f84706a7b4d6d68ede47e3)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:25.589772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:25.766992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e18569dc0f84706a7b4d6d68ede47e3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:25.767693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e18569dc0f84706a7b4d6d68ede47e3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:25.780537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e18569dc0f84706a7b4d6d68ede47e3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:25.787127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e18569dc0f84706a7b4d6d68ede47e3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:25.787811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e18569dc0f84706a7b4d6d68ede47e3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:25.844760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1125/2206] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:20:25 2022] DELETE /identity/v3/users/6e18569dc0f84706a7b4d6d68ede47e3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:20:25.849555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:25.850644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:25.850929 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:25.871483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:25.872695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:25.931381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1082/2207] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:25.941102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:25.942546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:25.997864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12912b1-a9a5-4763-9e5f-12f19d8c8f57 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:26.002553 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:26.003198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.003477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.003769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.013538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:26.014926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5309694-6df2-433e-885b-9b45b040e9cf service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:26.025138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1126/2208] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:20:26.436291 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:26.437816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:26.450118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216cac2-560b-4228-b862-08fd1df8f6fb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:26.454877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:26.455758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.456039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.456316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] PATH_INFO: `/v3/projects/ac430abda2c64e3fad2daec30d41778e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.457497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac430abda2c64e3fad2daec30d41778e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:26.460487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:26.576896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac430abda2c64e3fad2daec30d41778e', 'request_id': 'req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:26.627792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac430abda2c64e3fad2daec30d41778e', 'request_id': 'req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:26.633841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac430abda2c64e3fad2daec30d41778e', 'request_id': 'req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:26.644624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] The token cache is being invalidate because project ac430abda2c64e3fad2daec30d41778e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:20:26.644912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:26.645540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-35a2df7f-f1cd-48c9-9a06-96a6403c4bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:20:26.647950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1083/2209] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:20:26 2022] DELETE /identity/v3/projects/ac430abda2c64e3fad2daec30d41778e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:20:26.652818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.653302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.653408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.671160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:26.672336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:26.736489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1127/2210] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:26.744278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.744636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.744842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.764645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:26.765855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:26.777610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1084/2211] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:26.778866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:26.788688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b9718-ab6f-445b-9594-d420eb7ceb79 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:26.793814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:26.795084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.795084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.795466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.796217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:20:26.796530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:26.798151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:26.810563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1128/2212] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:20:26.814894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:26.824658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a643-9a34-4059-971f-340ef861c1fc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:26.829403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:26.830053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.830487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.830776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.831654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:26.835166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:26.921061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1085/2213] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:20:26 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:20:26.925346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:26.935586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b15a-f255-4bc7-b722-0ef9aab60d93 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:26.990023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:26.990729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:26.991013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:26.991289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:26.991997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:26.993904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:27.055797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1129/2214] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:20:26 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:20:27.060201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:27.070566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08186c0c-ff1a-4c30-89b7-9465c620a0f2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:27.075445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:27.076136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:27.076422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:27.076700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:27.077428 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:27.079063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:27.091453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1086/2215] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:20:27.093956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:27.104696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ad956-5086-4412-ae65-36e6392a0e9c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:27.109404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:27.110057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:27.110380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:27.110661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:27.111384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:27.112996 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b6dab-6dc1-4170-bf70-46629fd08b3a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:27.123533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1130/2216] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:20:27.127804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:27.137682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e36e5-c0f4-435a-9522-faca3c5f10c6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:27.142227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:27.143111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:27.143395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:27.143677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] PATH_INFO: `/v3/projects/8324b5c8a0ad43b09a459a73d73dcf85/users/943d9591fc444280b105a088b32f4d4e/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:27.158070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8324b5c8a0ad43b09a459a73d73dcf85, user_id=943d9591fc444280b105a088b32f4d4e, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:27.160764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:27.256773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1087/2217] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:20:27 2022] PUT /identity/v3/projects/8324b5c8a0ad43b09a459a73d73dcf85/users/943d9591fc444280b105a088b32f4d4e/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:20:27.262185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfc56-9650-4fb3-8823-72ffc8684be7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:27.262497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfc56-9650-4fb3-8823-72ffc8684be7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:27.262778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfc56-9650-4fb3-8823-72ffc8684be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:27.281379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-950bfc56-9650-4fb3-8823-72ffc8684be7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:27.282595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-950bfc56-9650-4fb3-8823-72ffc8684be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:27.340144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1131/2218] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:20:27.349672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:27.351421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:27.406763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db523c0d-8e79-4a28-aa04-9ac958f5d307 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:27.410446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:27.411077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:27.411357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:27.411631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:27.421226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:27.422665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:27.431996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1088/2219] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:20:33.436077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:33.437037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:33.437523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:33.468563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] MFA Rules not processed for user `943d9591fc444280b105a088b32f4d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:33.469778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:33.539777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1132/2220] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:33.544746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:33.555224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d938579-2606-4443-afc9-e245bf2d9864 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:33.559892 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:33.560571 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:33.560858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:33.561168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:33.561857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:33.563807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:33.749880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1089/2221] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:20:33 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:20:33.753728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:33.764935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0375a-6de9-4205-88cc-66e299858e43 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:33.830181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:33.831057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:33.831499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:33.832044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:33.832898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:33.839672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:33.935526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1133/2222] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:20:33 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:20:33.937046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:33.949871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4affd-eaba-462e-8f38-6173d6c735a2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:33.959341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:33.960196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:33.960486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:33.960764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:33.961497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:33.963155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:33.975655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1090/2223] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:20:33.979859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:33.990446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804ec6-ea7b-4a3e-a274-ef1d4920c768 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:33.995672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:33.996568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:33.996854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:33.997132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] PATH_INFO: `/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122/users/8bceba9754684137b58cbb6ea99c1b65/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:34.016466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad66bf5ec9a41cb90f8462c01ccc122, user_id=8bceba9754684137b58cbb6ea99c1b65, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:34.019954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:34.092096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1134/2224] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:20:33 2022] PUT /identity/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122/users/8bceba9754684137b58cbb6ea99c1b65/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:20:34.097323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:34.107305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67a366-b96a-4c53-a976-353ddd41ea0c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:34.164601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:34.165379 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:34.165738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:34.166085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:34.166923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:34.168846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42e21-127b-4a8d-b566-b52446d91d29 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:34.180501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1091/2225] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:20:34.184585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:34.195033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cdb23-801e-4578-91a3-6390b4f12ad0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:34.199837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:34.200720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:34.201008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:34.201287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] PATH_INFO: `/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122/users/8bceba9754684137b58cbb6ea99c1b65/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:34.203299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad66bf5ec9a41cb90f8462c01ccc122, user_id=8bceba9754684137b58cbb6ea99c1b65, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:34.205972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c0029-3e85-4825-9898-b1328705dd3b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:34.216958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1135/2226] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:20:34 2022] PUT /identity/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122/users/8bceba9754684137b58cbb6ea99c1b65/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:20:35.286836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341645-1506-485a-9dd3-310633212db9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:35.288358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341645-1506-485a-9dd3-310633212db9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:35.289803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341645-1506-485a-9dd3-310633212db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:35.312435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-4d341645-1506-485a-9dd3-310633212db9 None None] MFA Rules not processed for user `98be3f9cd80e4f62972dbf48591c7dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:35.312435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4d341645-1506-485a-9dd3-310633212db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:35.370347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1092/2227] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:40.556179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4d51d-0a8a-484f-bc47-447073f20016 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:40.556674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4d51d-0a8a-484f-bc47-447073f20016 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:40.556789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4d51d-0a8a-484f-bc47-447073f20016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:40.587078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7bb4d51d-0a8a-484f-bc47-447073f20016 None None] MFA Rules not processed for user `8bceba9754684137b58cbb6ea99c1b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:40.588339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7bb4d51d-0a8a-484f-bc47-447073f20016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:40.674501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1136/2228] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:20:40.687445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341645-1506-485a-9dd3-310633212db9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:40.688977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4d341645-1506-485a-9dd3-310633212db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:40.760669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341645-1506-485a-9dd3-310633212db9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:40.764425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:40.765301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:40.765600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:40.767458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:40.840263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:40.842511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:40.853228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1093/2229] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:20:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:20:41.087295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d942446-a1c3-446d-9f18-43e4b6dc55ad None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:41.087595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d942446-a1c3-446d-9f18-43e4b6dc55ad None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:41.087877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d942446-a1c3-446d-9f18-43e4b6dc55ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:41.109892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5d942446-a1c3-446d-9f18-43e4b6dc55ad None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:41.111195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5d942446-a1c3-446d-9f18-43e4b6dc55ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:41.182428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1137/2230] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:20:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:20:41.378969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:20:41.380837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:41.480549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffae7f-bedf-416e-93d9-eb5f7c402310 service glance] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:20:41.485936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:20:41.486619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:41.486904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:41.487237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:41.496806 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:20:41.498433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:20:41.508083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1094/2231] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:20:53.639508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:20:53.640078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:20:53.640192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:20:53.657441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] MFA Rules not processed for user `943d9591fc444280b105a088b32f4d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:20:53.658627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:20:53.666176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1138/2232] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:21:25.915048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:25.969360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113cb5-4579-4d89-874d-10b44a6d96ce service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:25.972973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:25.973614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:25.973898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:25.974174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:25.975946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:26.044524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:26.045956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:26.057176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1095/2233] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:28.528369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:28.540586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544dfe6-b7f5-4054-a491-3e7597f8dd8c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:28.545011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:28.545675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:28.546059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:28.546446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:28.552585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:28.567044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:28.568468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:28.580001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1139/2234] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:37.961898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:37.964013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:37.978350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339f8a-77d2-410f-a28b-891ed7d69352 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:37.983947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:37.984617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:37.984900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:37.985177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:37.986907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:38.046481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:38.048322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:38.057913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1096/2235] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:40.201802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:40.203242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:40.260368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7adb2-1aac-43be-924b-d7397061221d service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:40.263281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:40.263922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:40.264264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:40.264543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:40.266345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:40.324580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:40.326001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:40.335281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1140/2236] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:47.539446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:47.540811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:47.555008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337ccdf-7d57-4a0d-b81b-0f99d5cd9dcf service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:47.560382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:47.561008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:47.561288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:47.561559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:47.563271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:47.619907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:47.621498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:47.630563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1097/2237] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:49.370100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:49.378557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fce5-4ca5-4aaf-b3ac-655d086cea93 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:49.382141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:49.382812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:49.383093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:49.383368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:49.385721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:49.444333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:49.445757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0730d-4263-4665-8b79-6e267dc53aa8 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:49.455317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1141/2238] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:53.748306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:53.749791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:53.761381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb8672-e1dd-4f23-ad7c-5f33491f5a73 service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:53.765006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:53.765651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:53.765938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:53.766240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:53.768108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:53.778857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:53.780306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:53.788545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1098/2239] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:21:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:21:55.244635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:55.245399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:55.245399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:55.268121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:21:55.269543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:21:55.284078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1142/2240] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:21:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:21:55.420822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:21:55.433501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1f060-ed2a-4bc8-8788-0d949cfc6c88 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:21:55.445380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:21:55.446037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:21:55.447154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:21:55.447433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:21:55.459304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:21:55.460909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:21:55.474953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1099/2241] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:21:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:22:10.256737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:10.270135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6037c-f0d8-4d0b-8aeb-dc20053d77aa None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:10.276094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:10.277855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:10.278134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:10.278455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:10.280562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:10.293741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:10.295380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d16608-4d7d-4996-8cc6-7f1b7f58aee6 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:10.308878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1143/2242] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:13.444492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:13.457358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5c26c-2e86-491c-9019-6bbd5582a3ee service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:13.462447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:13.463187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:13.463556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:13.463921 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:13.466155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:13.544999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:13.546546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:13.559016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1100/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:13.580071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de0ae-8939-47c8-a078-99798860f769 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:13.580578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de0ae-8939-47c8-a078-99798860f769 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:13.580741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de0ae-8939-47c8-a078-99798860f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:13.599502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-990de0ae-8939-47c8-a078-99798860f769 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:22:13.600731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-990de0ae-8939-47c8-a078-99798860f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:13.608333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1144/2244] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:22:13.621201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:13.633005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ee98-3617-40c9-9a1d-9ed42c43e568 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:13.638896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:13.639550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:13.639837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:13.640118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:13.649109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:13.650718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:13.659361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1101/2245] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:22:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 14:22:16.602155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990de0ae-8939-47c8-a078-99798860f769 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:16.617548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990de0ae-8939-47c8-a078-99798860f769 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:16.624205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:16.625199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:16.625767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:16.626247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:16.628726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:16.646019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:16.648015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:16.664594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1145/2246] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:20.468453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:20.486096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504060ef-c0f1-4b26-9469-f85ece9c03f8 service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:20.504539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:20.505568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:20.506035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:20.506516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:20.510166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:20.526353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:20.527958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7df31-ee33-4303-8938-f29c62eac169 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:20.541909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1102/2247] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:32.047757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:32.061958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609311a1-14b3-4c02-be77-44230c43f1ed service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:32.065687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:32.066373 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.066666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.066944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.068712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:32.080104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:32.081537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c5f9e-8c5c-499d-a05c-777adb77490d service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:32.089846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1146/2248] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:32.396672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5a54-2886-42a4-91f1-6fd1bec0e174 None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.397591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5a54-2886-42a4-91f1-6fd1bec0e174 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.398148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf5a54-2886-42a4-91f1-6fd1bec0e174 None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.399913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1103/2249] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:22:32.406672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0738-2fc9-4c0b-8c84-ac2b04049cba None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.406961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0738-2fc9-4c0b-8c84-ac2b04049cba None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.407240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0738-2fc9-4c0b-8c84-ac2b04049cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.447137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bdee0738-2fc9-4c0b-8c84-ac2b04049cba None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:22:32.448345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bdee0738-2fc9-4c0b-8c84-ac2b04049cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:32.458412 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1147/2250] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:22:32.470690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf5a54-2886-42a4-91f1-6fd1bec0e174 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:32.482495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf5a54-2886-42a4-91f1-6fd1bec0e174 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:32.486352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:32.486996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.487277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.487551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.499393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:32.500814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1c082-5983-4f7c-95b6-c14056d992d0 service glance] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:32.511877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1104/2251] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:32.715831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c64e9-fd7d-49e4-b876-3c953aa049a2 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.716113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c64e9-fd7d-49e4-b876-3c953aa049a2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.716383 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c64e9-fd7d-49e4-b876-3c953aa049a2 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.717974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1148/2252] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:22:32.724451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8dfba-78c4-4fd0-8a32-d3e755171249 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.724738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8dfba-78c4-4fd0-8a32-d3e755171249 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.725014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8dfba-78c4-4fd0-8a32-d3e755171249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.750230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-eab8dfba-78c4-4fd0-8a32-d3e755171249 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:22:32.751589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-eab8dfba-78c4-4fd0-8a32-d3e755171249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:32.758235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1105/2253] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:22:32.775845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c64e9-fd7d-49e4-b876-3c953aa049a2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:32.788808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c64e9-fd7d-49e4-b876-3c953aa049a2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:32.792674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:32.793453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.796002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.796281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.805630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:32.807664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645a0a3-cbf4-434a-b1ca-628e3923539c service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:32.817670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1149/2254] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:32.915783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29a1b-1db2-463d-b124-467bd887e39c None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.916141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29a1b-1db2-463d-b124-467bd887e39c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.916417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a29a1b-1db2-463d-b124-467bd887e39c None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.917879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1106/2255] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:22:32.924421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.924785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.925138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.945193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:22:32.946503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:32.955701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1150/2256] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:22:32.966972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a29a1b-1db2-463d-b124-467bd887e39c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:32.977685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a29a1b-1db2-463d-b124-467bd887e39c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:32.981281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:32.982584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:32.982854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:32.983122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:32.993112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:32.994559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:33.006124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1107/2257] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:33.089603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:33.092110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:33.163369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8e9a0-04d3-452c-a471-835743221e6a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:33.170286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:33.171234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:33.171629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:33.171916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:33.174245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:33.187635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:33.189230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:33.199267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1151/2258] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:22:54.992310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:22:55.001254 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499731-95ac-4166-8e49-8581ac140cba service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:22:55.005214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:22:55.005840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:22:55.006121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:22:55.006425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:22:55.008138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:22:55.065378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:22:55.066907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:22:55.075871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1108/2259] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:22:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:24:59.296731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:24:59.315576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ecce7-a2cd-413d-9698-fddd0bb9c2c2 service placement] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:24:59.321273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:24:59.321947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:24:59.322262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:24:59.322548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:24:59.324538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:24:59.336719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:24:59.338388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:24:59.348971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1152/2260] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:24:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:25:33.211657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:25:33.223953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6013b3-bb2d-4f84-8abd-a728c111c149 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:25:33.229445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:25:33.230117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:25:33.230788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:25:33.231072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:25:33.232898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:25:33.247399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:25:33.249038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:25:33.260648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1109/2261] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:25:55.488608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:25:55.491405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:25:55.505946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da8f8d-a02d-4f06-bcbc-2c0cb4eeb228 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:25:55.510516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:25:55.511165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:25:55.511449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:25:55.511726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:25:55.523049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:25:55.524530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:25:55.540337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1153/2262] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:26.381971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:26.392628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34a45d-f9e6-47e6-83c0-bf88bd7ed5a6 service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:26.396154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:26.396847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:26.397150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:26.397423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:26.408628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:26.410036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:26.421505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1110/2263] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:28.483456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:28.494303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56954-f639-4c45-8d31-04fbfdb89783 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:28.498766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:28.499409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:28.499691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:28.499994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:28.511292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:28.512710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:28.523457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1154/2264] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:26:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:38.184480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:38.193119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b14cc-b633-4f25-8171-43d8320297b9 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:38.197926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:38.198645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:38.198923 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:38.199196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:38.209192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:38.212918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:38.240323 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1111/2265] 10.222.0.58 () {60 vars in 1396 bytes} [Tue May 10 14:26:38 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:58.780185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:58.790638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69460d-5713-4755-a6dd-3c125ac06461 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:58.794293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:58.794938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:58.795223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:58.795502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:58.804007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:58.806387 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:58.834067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1155/2266] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:58.894645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:58.903748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949dbdb-a354-4b6d-9a08-59dd1b36716e service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:58.907306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:58.907946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:58.908230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:58.908500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:58.917334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:58.918790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2075249-fdb5-4661-b3aa-54151d759719 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:58.946948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1112/2267] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:26:58.994096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:26:59.002618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7f9e-6140-49e2-8122-0fd7335f46c1 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:26:59.006123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:26:59.006808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:26:59.007084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:26:59.007356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:26:59.015782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:26:59.017193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:26:59.044248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1156/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:27:07.127862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:07.128380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:07.128380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:07.167075 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] MFA Rules not processed for user `ecb4c97543e1452d8e6c8f1564addc76`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:27:07.168193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:27:07.174968 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1113/2269] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:27:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:27:07.475070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:27:07.485402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8831-e9e1-4873-95c5-d455510bd203 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:27:07.489135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:27:07.489771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:07.490052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:07.490362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:07.492120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:27:07.501499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:27:07.502960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:27:07.511030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1157/2270] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:27:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:27:10.814821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:27:10.824586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1edc4-c225-49b5-8661-4c216aaf54af None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:27:10.828108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:27:10.828744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:10.838042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:10.838042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:10.838042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:27:10.838770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:27:10.847358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1114/2271] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:27:11.746797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:27:11.762378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf3332-78c0-43ca-8a31-ad1dee9b2a21 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:27:11.767859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:27:11.768815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:11.769113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:11.770835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:11.780749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:27:11.782452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:27:11.796135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1158/2272] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:27:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:27:22.127564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:27:22.140604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42a913-aff5-4426-b7a6-6a77544b81a6 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:27:22.146592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:27:22.147246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:22.147533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:22.147811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:22.157340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:27:22.158969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:27:22.170493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1115/2273] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:27:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:27:56.006739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:27:56.017111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9929fd-81c4-42a1-8953-c9c42370b863 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:27:56.020721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:27:56.021353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:27:56.021631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:27:56.021899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:27:56.031095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:27:56.032505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:27:56.060973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1159/2274] 10.222.0.58 () {60 vars in 1396 bytes} [Tue May 10 14:27:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:30:39.209403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:30:39.211974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:30:39.303180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb5b09-7e11-47ad-afd6-2e9464b7f0dc service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:30:39.308501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:30:39.309154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:30:39.309436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:30:39.309711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:30:39.319442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:30:39.321037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:30:39.332551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1116/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:30:59.827884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:30:59.861709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013f06-d6b4-469e-be6c-d2d3b4d22062 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:30:59.865657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:30:59.866336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:30:59.866622 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:30:59.866929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:30:59.869001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:30:59.950131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:30:59.951606 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:30:59.970863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1160/2276] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:30:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:31:00.164348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:31:00.203330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316297-5d07-4606-97ed-38e7635ce67e service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:31:00.208905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:31:00.209573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:31:00.209859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:31:00.210137 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:31:00.277327 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:31:00.278909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:31:00.298081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1117/2277] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:31:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:31:57.230854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:31:57.293505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5a03-a508-46b6-965a-ee21f0cb7963 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:31:57.297462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:31:57.298132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:31:57.298414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:31:57.298689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:31:57.300634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:31:57.387226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:31:57.389173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:31:57.400088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1161/2278] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:31:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:31:57.408086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:31:57.417058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f961a-e19a-4907-8db1-0e7e2c61746d service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:31:57.420653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:31:57.421278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:31:57.421557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:31:57.421830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:31:57.423774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:31:57.494424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:31:57.495844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:31:57.505289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1118/2279] 10.222.0.58 () {60 vars in 1396 bytes} [Tue May 10 14:31:57 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:31:57.737256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:31:57.745728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0c15d-6205-4db2-a255-d40221a7dd29 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:31:57.750107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:31:57.750794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:31:57.751076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:31:57.751351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:31:57.753221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:31:57.767590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:31:57.770018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:31:57.780435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1162/2280] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:31:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:32:39.857378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:32:39.859818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:32:39.941416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933be2b-8fba-42fd-958f-658cd4caac1d service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:32:39.945069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:32:39.945726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:32:39.946008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:32:39.946314 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:32:39.948246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:32:39.959287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:32:39.960732 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:32:39.969449 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1119/2281] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:33:00.632856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:33:00.633521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:00.668937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc134e1-2f8b-4449-aacc-d192a0d9b9f4 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:33:00.672886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:33:00.673552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:33:00.673824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:33:00.674096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:33:00.676163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:00.763831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:33:00.765300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:33:00.780105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1163/2282] 10.222.0.58 () {60 vars in 1396 bytes} [Tue May 10 14:33:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:33:00.880939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:33:00.892001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a8ec2-6ac2-41e3-b356-b98125fbade1 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:33:00.896041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:33:00.896753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:33:00.897108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:33:00.897603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:33:00.899818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:00.940220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:33:00.941830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:00.957992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04fef-837c-452f-b04f-59db036af140 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:33:00.964477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:33:00.965120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:33:00.965399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:33:00.965678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:33:00.967636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:00.980890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:33:00.982493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:33:00.992372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1120/2283] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:33:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:33:01.044329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:33:01.045951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0dc47-589e-451c-b02e-e871dffb513d service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:33:01.056401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1164/2284] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:33:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:33:01.088942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:33:01.099572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb1df-3eab-4ad8-8099-2c01439f303a service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:33:01.104226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:33:01.104890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:33:01.105175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:33:01.105484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:33:01.107734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:33:01.171014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:33:01.172458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:33:01.181766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1121/2285] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:33:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:02.633825 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:02.634604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:02.634604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:02.710879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:35:02.712255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:02.799472 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1165/2286] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 10 14:35:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:35:02.972434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:02.972434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:03.073797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f4ba-6437-4685-a9f4-a52d2fa013da service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:03.081673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:03.082395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:03.082672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:03.082943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:03.092897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:03.095394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd4d82-312c-4020-b1dc-378b91bf43ca service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:03.107002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1122/2287] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:35:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:35:05.877598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:05.894039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11f11-2152-473f-a7d6-cbfa12b5de3a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:05.900956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:05.901779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:05.902160 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:05.902565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:05.913684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:05.929813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:05.931978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9960c9-6404-43d8-92e5-03e3e24f2ca8 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:05.946991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1166/2288] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:35:06.165499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:06.166226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:06.166226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:06.167578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1123/2289] 10.222.0.58 () {54 vars in 912 bytes} [Tue May 10 14:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 10 14:35:06.173632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4f3f8-22a7-4d1f-ac77-b17ca41a6bbe None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:06.174034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4f3f8-22a7-4d1f-ac77-b17ca41a6bbe None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:06.174401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4f3f8-22a7-4d1f-ac77-b17ca41a6bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:06.222841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d5b4f3f8-22a7-4d1f-ac77-b17ca41a6bbe None None] MFA Rules not processed for user `639566a1a9de49ef841486e9e21a2116`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:35:06.224141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d5b4f3f8-22a7-4d1f-ac77-b17ca41a6bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:06.303602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1167/2290] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:35:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:35:06.330185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:06.332048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:06.349441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b4e83-54b1-4ab1-9c82-d1731098afdb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:06.354855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:06.355502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:06.355779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:06.356051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:06.365734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:06.367339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:06.377694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1124/2291] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:10.920261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:10.920261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:10.920261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:10.946570 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] MFA Rules not processed for user `9dae4d7a57f343c8bb7b92dbebea181d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:35:10.947821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:10.956771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1168/2292] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:35:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:35:11.286233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:11.287898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:11.300217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f6d5d-a35a-411d-9f3b-cd05c7f653ef service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:11.304093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:11.304743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:11.305026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:11.305298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:11.313156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:11.314693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:11.323012 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1125/2293] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:11.840832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:11.854951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d78f0-317e-4c62-b2d0-b0ba574f046c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:11.861905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:11.862631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:11.862915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:11.863192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:11.865112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:11.880028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:11.881633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:11.892788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1169/2294] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:16.677476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:16.679789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:16.697828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7f0b-f8e8-4b3a-8501-67119ea2085e service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:16.709288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:16.709954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:16.710497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:16.710783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:16.712778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:16.729441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:16.731178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:16.743480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1126/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:22.071929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:22.085459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6507c-95ed-4939-83f1-1fb69e06449b service placement] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:22.092190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:22.092847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:22.093182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:22.093465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:22.095538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:22.110327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:22.111911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:22.127252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1170/2296] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:35:42.144802 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:35:42.156299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d9307-b33f-4c0a-823d-85d27800d98f service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:35:42.161583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:35:42.162435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:35:42.164223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:35:42.164709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:35:42.166898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:35:42.180167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:35:42.182029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:35:42.196986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1127/2297] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:35:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:36:00.240766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:00.250395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a5fa-bd9d-4874-bcad-056c2265339a service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:00.254715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:00.255371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:00.255647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:00.255924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:00.257862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:00.283356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:00.284794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d2cc-04aa-441f-afbb-1e1d5d9ff6bd service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:00.294523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1171/2298] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:36:00.349230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:00.365998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227d92-8681-46c0-8175-610d5cea66ff service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:00.372951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:00.373654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:00.373941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:00.374251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:00.384308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:00.386719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:00.399301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1128/2299] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:36:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:36:00.576718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:00.576718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:00.576718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:00.588305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:36:00.589592 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:00.597036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1172/2300] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 10 14:36:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:36:00.612818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:00.625917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029d1ef-63c6-4d2e-8366-9a1c364a7f03 service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:00.631425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:00.632081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:00.632371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:00.632647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:00.641678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:00.643667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:00.652762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1129/2301] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:36:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 14:36:02.201730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:02.211149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec671dc-d633-497a-9b8f-371f6fd9fd43 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:02.214734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:02.215371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:02.215651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:02.215926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:02.217813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:02.303541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:02.305059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:02.324156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1173/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:36:11.177931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:11.179570 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:11.193890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b3fef-6c47-465c-88bd-71e749b3df4c service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:11.198673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:11.200103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:11.200397 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:11.200667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] PATH_INFO: `/v3/users/943d9591fc444280b105a088b32f4d4e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:11.202990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=943d9591fc444280b105a088b32f4d4e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:11.204954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:11.448809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943d9591fc444280b105a088b32f4d4e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:11.449483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943d9591fc444280b105a088b32f4d4e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:11.461609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943d9591fc444280b105a088b32f4d4e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:11.462276 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943d9591fc444280b105a088b32f4d4e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:11.468889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943d9591fc444280b105a088b32f4d4e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:11.560477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1130/2303] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:36:11 2022] DELETE /identity/v3/users/943d9591fc444280b105a088b32f4d4e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:36:11.837743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:11.851576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762748b3-8c1f-475c-82fd-b5ab2d7b277f service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:11.924802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:11.926545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:11.926545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:11.926545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] PATH_INFO: `/v3/projects/8324b5c8a0ad43b09a459a73d73dcf85` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:11.927796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8324b5c8a0ad43b09a459a73d73dcf85)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:11.929904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:12.075552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8324b5c8a0ad43b09a459a73d73dcf85', 'request_id': 'req-ce0535d8-b553-482c-b9a6-d99b0a731012'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.190798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8324b5c8a0ad43b09a459a73d73dcf85', 'request_id': 'req-ce0535d8-b553-482c-b9a6-d99b0a731012'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.196732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8324b5c8a0ad43b09a459a73d73dcf85', 'request_id': 'req-ce0535d8-b553-482c-b9a6-d99b0a731012'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.208247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] The token cache is being invalidate because project 8324b5c8a0ad43b09a459a73d73dcf85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:36:12.208539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.209212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.210882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1174/2304] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:36:11 2022] DELETE /identity/v3/projects/8324b5c8a0ad43b09a459a73d73dcf85 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:36:12.215347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:12.216807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:12.231228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4405-a54f-48db-aa2b-873c983c8f54 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:12.292568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:12.300438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:12.300438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:12.300438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] PATH_INFO: `/v3/users/8bceba9754684137b58cbb6ea99c1b65` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:12.310109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bceba9754684137b58cbb6ea99c1b65)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:12.312406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:12.618704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bceba9754684137b58cbb6ea99c1b65'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.619363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bceba9754684137b58cbb6ea99c1b65'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.631505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bceba9754684137b58cbb6ea99c1b65'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.632165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bceba9754684137b58cbb6ea99c1b65'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.639013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bceba9754684137b58cbb6ea99c1b65'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:12.727144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1131/2305] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:36:12 2022] DELETE /identity/v3/users/8bceba9754684137b58cbb6ea99c1b65 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:36:12.996519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:12.998127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:13.017116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0535d8-b553-482c-b9a6-d99b0a731012 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:13.081961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:13.082840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:13.083120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:13.083395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] PATH_INFO: `/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:13.094384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad66bf5ec9a41cb90f8462c01ccc122)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:13.096490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:13.265532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad66bf5ec9a41cb90f8462c01ccc122', 'request_id': 'req-718a83dc-e809-4b39-8120-bd79346591b7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:13.382820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad66bf5ec9a41cb90f8462c01ccc122', 'request_id': 'req-718a83dc-e809-4b39-8120-bd79346591b7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:13.389194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad66bf5ec9a41cb90f8462c01ccc122', 'request_id': 'req-718a83dc-e809-4b39-8120-bd79346591b7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:13.402835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] The token cache is being invalidate because project bad66bf5ec9a41cb90f8462c01ccc122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:36:13.403135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:13.403839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:36:13.405386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1175/2306] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:36:12 2022] DELETE /identity/v3/projects/bad66bf5ec9a41cb90f8462c01ccc122 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:36:58.076331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:36:58.077872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:58.135653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d243919-d801-45a6-9377-f5a33934c446 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:36:58.184102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:36:58.184762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:36:58.185043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:36:58.185319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:36:58.187122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:36:58.252516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:36:58.253873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:36:58.272072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1132/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:36:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:37:00.697335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:37:00.699644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:00.767536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a83dc-e809-4b39-8120-bd79346591b7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:37:00.771132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:37:00.771776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:37:00.772055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:37:00.772354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:37:00.774323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:00.845641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:37:00.847210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a6f51-6e55-43f9-9bfc-a5f43429d7ba service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:37:00.859312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1176/2308] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:37:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:37:02.034277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:37:02.043999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca768648-ff5e-4015-854c-91d8fc1cee65 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:37:02.047774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:37:02.048438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:37:02.048719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:37:02.049001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:37:02.050804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:02.065675 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:37:02.067414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:37:02.080309 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1133/2309] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:37:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:37:08.322228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:37:08.322760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:37:08.322884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:37:08.370991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] MFA Rules not processed for user `ecb4c97543e1452d8e6c8f1564addc76`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:37:08.372124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:08.435985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1177/2310] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:37:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:37:47.796952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:37:47.798431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:47.809962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4912b-6214-4b6d-b92a-15ad6915186a service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:37:47.813824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:37:47.814514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:37:47.814799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:37:47.815077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:37:47.816990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:37:47.827589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:37:47.828956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:37:47.837706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1134/2311] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:37:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:38:01.327238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:38:01.328772 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:01.341328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5efb-6642-49c4-89ba-c62b432cd945 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:38:01.344829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:38:01.345459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:38:01.345739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:38:01.346012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:38:01.348866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:01.417081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:38:01.418573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:38:01.436418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1178/2312] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:38:01.518794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:38:01.528993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf585-ea1a-4494-92fb-876b84a6980b service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:38:01.532812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:38:01.533390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:38:01.533670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:38:01.533944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:38:01.536678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:01.603715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:38:01.605142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:38:01.623273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1135/2313] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:38:01.849561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:38:01.858102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b536e-8ea5-4ca4-a7e3-6ebdd93cc75a service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:38:01.861725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:38:01.862651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:38:01.862936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:38:01.863211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:38:01.864993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:01.937070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:38:01.938550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:38:01.957908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1179/2314] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:38:39.878677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:38:39.880071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:39.896322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8f1db-7745-4b1e-affa-8775b1b72936 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:38:39.901723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:38:39.902400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:38:39.902684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:38:39.902960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:38:39.904667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:38:39.916208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:38:39.917869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:38:39.927577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1136/2315] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:38:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:40:47.077563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:40:47.079161 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:40:47.206701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4db8-558c-445b-be08-aef35b4dd283 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:40:47.212069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:40:47.212720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:40:47.213001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:40:47.213278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:40:47.215067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:40:47.227114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:40:47.228701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:40:47.238962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1180/2316] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:40:59.874600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:40:59.905740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbb53-dcef-4bd3-bc40-f7e957d96d0a service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:40:59.909651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:40:59.910393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:40:59.910679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:40:59.910956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:40:59.913522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:40:59.973433 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:40:59.974847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:40:59.984926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1137/2317] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:41:03.681438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:03.702337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129343db-c8b8-4ce3-ad00-fd91b376d7ea service placement] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:03.706030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:03.706718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:03.706998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:03.707270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:03.709047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:03.775423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:03.776874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:03.786756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1181/2318] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:41:12.367163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:12.368674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:12.386415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d9e5c-640c-4b63-a343-a1cf3882db3c service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:12.391886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:12.392761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:12.393170 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:12.393459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] PATH_INFO: `/v3/users/98be3f9cd80e4f62972dbf48591c7dd4` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:12.394711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98be3f9cd80e4f62972dbf48591c7dd4)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:12.396631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:12.594794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98be3f9cd80e4f62972dbf48591c7dd4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:12.597100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98be3f9cd80e4f62972dbf48591c7dd4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:12.611127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98be3f9cd80e4f62972dbf48591c7dd4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:12.611882 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98be3f9cd80e4f62972dbf48591c7dd4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:12.620053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98be3f9cd80e4f62972dbf48591c7dd4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:12.684998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1138/2319] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:41:12 2022] DELETE /identity/v3/users/98be3f9cd80e4f62972dbf48591c7dd4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:41:12.948742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:12.960235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da6ba-6061-4bef-98c0-c1686be74271 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:13.037329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:13.039076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:13.039452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:13.039736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] PATH_INFO: `/v3/projects/674aa2ca44674c908e6393ef25d6719a` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:13.052114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=674aa2ca44674c908e6393ef25d6719a)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:13.055411 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:13.175214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674aa2ca44674c908e6393ef25d6719a', 'request_id': 'req-c47938a4-d24d-4cb4-b053-08926184d9ff'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.248488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674aa2ca44674c908e6393ef25d6719a', 'request_id': 'req-c47938a4-d24d-4cb4-b053-08926184d9ff'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.255334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674aa2ca44674c908e6393ef25d6719a', 'request_id': 'req-c47938a4-d24d-4cb4-b053-08926184d9ff'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.269668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] The token cache is being invalidate because project 674aa2ca44674c908e6393ef25d6719a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:41:13.269960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.270673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.272810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1182/2320] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:41:12 2022] DELETE /identity/v3/projects/674aa2ca44674c908e6393ef25d6719a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:41:13.277028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:13.278497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:13.291862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f92b-95dc-4369-b12f-d99f8fa9ab22 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:13.364492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:13.365569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:13.366028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:13.366389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] PATH_INFO: `/v3/users/5230438fb71d417bb5142bd09b160607` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:13.380607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5230438fb71d417bb5142bd09b160607)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:13.383069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:13.578945 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5230438fb71d417bb5142bd09b160607'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.579949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5230438fb71d417bb5142bd09b160607'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.596167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5230438fb71d417bb5142bd09b160607'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.596897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5230438fb71d417bb5142bd09b160607'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.604947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5230438fb71d417bb5142bd09b160607'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:13.712505 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1139/2321] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:41:13 2022] DELETE /identity/v3/users/5230438fb71d417bb5142bd09b160607 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:41:13.941597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:13.948141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:13.971445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47938a4-d24d-4cb4-b053-08926184d9ff admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:14.027768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:14.028620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:14.028900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:14.029169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] PATH_INFO: `/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:14.039583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228ca61d6fec4e0397cdfc00f0126ed5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:14.041512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:14.167936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228ca61d6fec4e0397cdfc00f0126ed5', 'request_id': 'req-20bf692c-64fd-4c07-bc0b-933592df23c3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:14.265441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228ca61d6fec4e0397cdfc00f0126ed5', 'request_id': 'req-20bf692c-64fd-4c07-bc0b-933592df23c3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:14.271178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228ca61d6fec4e0397cdfc00f0126ed5', 'request_id': 'req-20bf692c-64fd-4c07-bc0b-933592df23c3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:14.282686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] The token cache is being invalidate because project 228ca61d6fec4e0397cdfc00f0126ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:41:14.282977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:14.283619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:41:14.285169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1183/2322] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:41:13 2022] DELETE /identity/v3/projects/228ca61d6fec4e0397cdfc00f0126ed5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:41:57.065595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:41:57.067061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:57.123483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4965a9f-52c9-4408-86c5-e9840f05f871 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:41:57.170890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:41:57.171662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:41:57.171960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:41:57.172234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:41:57.173985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:41:57.231955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:41:57.233426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:41:57.243991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1140/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:41:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:42:00.337591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:42:00.339102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:00.394410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf692c-64fd-4c07-bc0b-933592df23c3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:42:00.398111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:42:00.398788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:42:00.399069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:42:00.399343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:42:00.401111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:00.492676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:42:00.494102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:42:00.513701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1184/2324] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:42:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:42:01.685418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:42:01.695188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2598b-c66a-4589-bfe8-a0e051ea0bf1 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:42:01.698919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:42:01.699559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:42:01.700052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:42:01.700332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:42:01.702232 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:01.716375 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:42:01.718104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:42:01.730422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1141/2325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:42:46.111342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:42:46.113621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:46.127600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de094-4881-41bb-9cac-de130dbf88c6 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:42:46.133522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:42:46.134166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:42:46.134478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:42:46.134753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:42:46.136566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:46.150043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:42:46.151807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:42:46.164048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1185/2326] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:42:54.567931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:42:54.569367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:54.602670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a76a0-d892-477b-9a12-c3d94bf4e3bd service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:42:54.607004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:42:54.607706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:42:54.608139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:42:54.608426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:42:54.610320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:42:54.621208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:42:54.622719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:42:54.633297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1142/2327] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:42:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:01.538180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:01.549376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4af12-bfa1-47d8-9639-3eda5f8a57a2 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:01.556536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:01.557193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:01.557507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:01.557788 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:01.559658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:01.634761 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:01.636432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:01.645762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1186/2328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:01.815356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:01.817241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:01.836633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c321c-ed32-49d2-919e-08792cf417dc service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:01.842477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:01.843133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:01.843414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:01.843685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:01.845436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:01.929155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:01.930797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:01.940966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1143/2329] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:02.168941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:02.178595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42ced-5b62-45a7-b3e6-2b054f374b28 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:02.182243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:02.182895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:02.183176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:02.183452 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:02.185450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:02.188721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:02.203138 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4e4c7-af6a-4234-a33d-74a0b8d33417 service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:02.210036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:02.210787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:02.211071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:02.211349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:02.214379 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:02.278831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:02.279784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:02.281382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:02.283325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:02.296297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1144/2330] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:02.303024 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1187/2331] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:03.753905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:03.770654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c77d37-1caa-452c-9890-bf0b680c80b0 service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:03.778393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:03.779420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:03.779796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:03.780169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:03.782447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:03.798764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:03.802372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:03.824711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1145/2332] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:08.294615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:08.303977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02945d-21c0-4295-9ddb-d7178881d2d2 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:08.308410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:08.309010 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:08.309308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:08.309597 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:08.311756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:08.384499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:08.385975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:08.396662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1188/2333] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:12.730081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:12.731580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:12.749403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3482-8f47-465b-919f-3bda26c05f1c service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:12.757826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:12.758743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:12.759027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:12.759306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] PATH_INFO: `/v3/users/5e3edc87e7014d8598ba8c9517b81cf7` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:12.760502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3edc87e7014d8598ba8c9517b81cf7)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:12.762465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:13.011842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3edc87e7014d8598ba8c9517b81cf7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.012490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3edc87e7014d8598ba8c9517b81cf7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.026070 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3edc87e7014d8598ba8c9517b81cf7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.026766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3edc87e7014d8598ba8c9517b81cf7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.033720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-07dec7d0-289d-4ae8-b31c-f430e0f53d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3edc87e7014d8598ba8c9517b81cf7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.156872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1146/2334] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:43:12 2022] DELETE /identity/v3/users/5e3edc87e7014d8598ba8c9517b81cf7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:43:13.514691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:13.529048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44b1a4-adcd-4960-a703-606241501f16 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:13.597491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:13.598446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:13.598730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:13.599002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] PATH_INFO: `/v3/projects/b8a7f6dc00b5487b9c323783ef5aaa2b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:13.600224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a7f6dc00b5487b9c323783ef5aaa2b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:13.602137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:13.746531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a7f6dc00b5487b9c323783ef5aaa2b', 'request_id': 'req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.831742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a7f6dc00b5487b9c323783ef5aaa2b', 'request_id': 'req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.838233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a7f6dc00b5487b9c323783ef5aaa2b', 'request_id': 'req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.850705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] The token cache is being invalidate because project b8a7f6dc00b5487b9c323783ef5aaa2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:43:13.851658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.851770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-60b61a14-1d99-4cfb-80f8-a3ebf3fc253b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:43:13.853314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1189/2335] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:43:13 2022] DELETE /identity/v3/projects/b8a7f6dc00b5487b9c323783ef5aaa2b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:43:13.860177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:13.861121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:13.861121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:13.920949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:43:13.922811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:14.014010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1147/2336] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:43:14.016487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.016758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.017045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.037980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:43:14.039400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:14.048709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1190/2337] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:43:14.054176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.064481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ae96b-72c1-4909-a34f-e7d5e5351204 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.071873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.072628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.072966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.073249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.074002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:43:14.074347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.075963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.089304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1148/2338] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:43:14.093914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.104089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c65f0-9610-4bba-9797-3e50324e4d40 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.108975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.109632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.109917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.110218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.111182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.113893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.289531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1191/2339] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:43:14 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:43:14.293700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.303957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b539cf-b806-4ec3-9030-1e3dbf7cb089 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.359731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.360406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.360944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.361229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.361967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.363978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.505985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1149/2340] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:43:14 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:43:14.510565 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.521487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32d33-bbe5-4a52-8f97-c0512eaa5498 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.526085 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.526780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.527062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.527335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.528065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.529746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.540377 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1192/2341] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:43:14.544797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.555906 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2602b7-75cc-4cc7-b749-e1bf5fae2203 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.560908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.561622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.561909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.562185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.562974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.564610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca837d1c-1786-462d-a6a2-5961574c18d0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.575112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1150/2342] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:43:14.579360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.590601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de559e6-e17a-4264-8349-ee3067ec08b0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.595414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.596168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.596455 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.597299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] PATH_INFO: `/v3/projects/2d6fe7b6f78742e29bf322a4b6bfcc28/users/5689bdcdb5934536bb7844d79bd833fd/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.613326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6fe7b6f78742e29bf322a4b6bfcc28, user_id=5689bdcdb5934536bb7844d79bd833fd, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.616067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:14.734735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1193/2343] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:43:14 2022] PUT /identity/v3/projects/2d6fe7b6f78742e29bf322a4b6bfcc28/users/5689bdcdb5934536bb7844d79bd833fd/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:43:14.746444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.747904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.747904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.776551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] MFA Rules not processed for user `5689bdcdb5934536bb7844d79bd833fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:43:14.779628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:14.856787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1151/2344] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:43:14.861503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:14.925774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2708d-fdd1-4f81-aaf0-aea21a0127df admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:14.930620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:14.931290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:14.931564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:14.931841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:14.932541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:14.934488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.074116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1194/2345] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:43:14 2022] POST /identity/v3/projects => generated 369 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:43:15.080791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:15.102873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2b7b7-de5c-47a2-aea0-25f3f9ab1d0d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:15.189793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:15.191465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.191677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.191989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.192829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:15.194951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.335292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1152/2346] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:43:15 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:43:15.340042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:15.356263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e137b-b614-4825-957d-22e48482b292 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:15.364221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:15.365038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.365359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.365698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.366779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:15.368847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.384051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1195/2347] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:43:15.388444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:15.402679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2788e-90d9-49da-b11b-723b71de171e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:15.410713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:15.413418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.413971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.414430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] PATH_INFO: `/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0/users/34b2542baa9d457991d79574b0549253/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.438642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6385e1aaf7c94db3a3e144a68c1be1c0, user_id=34b2542baa9d457991d79574b0549253, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:15.443178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.555723 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1153/2348] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:43:15 2022] PUT /identity/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0/users/34b2542baa9d457991d79574b0549253/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:43:15.559946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:15.570508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0768def-e3b5-490e-b320-ca269e8914cf admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:15.627022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:15.627683 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.627968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.628246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.628972 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:15.630615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9b4e7-87be-4cfc-9526-5b5517404c73 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.641295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1196/2349] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:43:15.645547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:15.656435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72008b2c-800e-4814-9756-4eecd5300797 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:15.661352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:15.662292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.662578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.662988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] PATH_INFO: `/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0/users/34b2542baa9d457991d79574b0549253/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.665008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6385e1aaf7c94db3a3e144a68c1be1c0, user_id=34b2542baa9d457991d79574b0549253, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:15.667698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:15.678676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1154/2350] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:43:15 2022] PUT /identity/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0/users/34b2542baa9d457991d79574b0549253/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:43:15.690876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:15.691611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:15.691611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:15.719208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] MFA Rules not processed for user `34b2542baa9d457991d79574b0549253`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:43:15.720433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:15.791635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1197/2351] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:43:59.263439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:59.265200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:59.336444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e370-9609-4da5-b79c-10d9970083c6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:59.340200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:59.340840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:59.341118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:59.341389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:59.343158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:43:59.400040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:59.401435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:59.411062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1155/2352] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:43:59.491471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:43:59.500894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454448-5d3f-4d38-96ca-dc7ced324cb9 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:43:59.504493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:43:59.505128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:43:59.505406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:43:59.505676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:43:59.514734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:43:59.516159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:43:59.525872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1198/2353] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:43:59 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:45:52.052096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:45:52.053513 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:45:52.123260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c551c5-4614-4612-a5df-34b225f53b82 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:45:52.128196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:45:52.128843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:45:52.129129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:45:52.129409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:45:52.131300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:45:52.142682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:45:52.144247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a2c93-13a4-4871-96e9-c38c5d69d568 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:45:52.154606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1156/2354] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:46:01.095148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:01.163654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5844db-1c92-45e5-bf2d-e4819e51d4a5 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:01.169480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:01.170392 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:01.170682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:01.171275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] PATH_INFO: `/v3/users/5689bdcdb5934536bb7844d79bd833fd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:01.172368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5689bdcdb5934536bb7844d79bd833fd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:01.174372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:01.405281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5689bdcdb5934536bb7844d79bd833fd'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:01.405974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5689bdcdb5934536bb7844d79bd833fd'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:01.418981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5689bdcdb5934536bb7844d79bd833fd'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:01.425816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5689bdcdb5934536bb7844d79bd833fd'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:01.429460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5689bdcdb5934536bb7844d79bd833fd'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:01.520172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1199/2355] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:46:01 2022] DELETE /identity/v3/users/5689bdcdb5934536bb7844d79bd833fd => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:01.524478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:01.524989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:01.525105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:01.542187 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:46:01.544957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:01.606263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1157/2356] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:46:01.616183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:01.617604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:01.677756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004457-b88a-4fdc-b822-db0eda8594de admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:01.682123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:01.682803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:01.683140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:01.684684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:01.694740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:01.696375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:01.707589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1200/2357] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:46:02.179293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:02.179957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:02.193401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f96b0-5382-4fb1-b5b6-e2953a443dbd None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:02.198267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:02.199777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:02.200059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:02.200329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] PATH_INFO: `/v3/projects/2d6fe7b6f78742e29bf322a4b6bfcc28` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:02.201788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6fe7b6f78742e29bf322a4b6bfcc28)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:02.205124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:02.320381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6fe7b6f78742e29bf322a4b6bfcc28', 'request_id': 'req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.450950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6fe7b6f78742e29bf322a4b6bfcc28', 'request_id': 'req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.457102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6fe7b6f78742e29bf322a4b6bfcc28', 'request_id': 'req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.482459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] The token cache is being invalidate because project 2d6fe7b6f78742e29bf322a4b6bfcc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:46:02.482748 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.483412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.484949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1158/2358] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:46:02 2022] DELETE /identity/v3/projects/2d6fe7b6f78742e29bf322a4b6bfcc28 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:02.489195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:02.517385 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b0f46-c8d8-4530-9fd4-f3a1eaee4600 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:02.571926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:02.572790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:02.573071 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:02.573350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] PATH_INFO: `/v3/users/34b2542baa9d457991d79574b0549253` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:02.574636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b2542baa9d457991d79574b0549253)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:02.576554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:02.790926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b2542baa9d457991d79574b0549253'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.791585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b2542baa9d457991d79574b0549253'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.803522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b2542baa9d457991d79574b0549253'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.810229 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b2542baa9d457991d79574b0549253'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.810869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8bd1872e-2328-43fe-bcc8-d55bd7f26e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b2542baa9d457991d79574b0549253'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:02.906753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1201/2359] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:46:02 2022] DELETE /identity/v3/users/34b2542baa9d457991d79574b0549253 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:03.335622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:03.337071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:03.363191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb40ccf-bcbd-4826-83ca-1a5f5ae3be82 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:03.421163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:03.422025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:03.422343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:03.422621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] PATH_INFO: `/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:03.423809 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6385e1aaf7c94db3a3e144a68c1be1c0)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:03.425702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:03.545247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6385e1aaf7c94db3a3e144a68c1be1c0', 'request_id': 'req-290daf0a-2320-4c33-b91f-7e7ddb287e15'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:03.614492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6385e1aaf7c94db3a3e144a68c1be1c0', 'request_id': 'req-290daf0a-2320-4c33-b91f-7e7ddb287e15'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:03.620279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6385e1aaf7c94db3a3e144a68c1be1c0', 'request_id': 'req-290daf0a-2320-4c33-b91f-7e7ddb287e15'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:03.631534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] The token cache is being invalidate because project 6385e1aaf7c94db3a3e144a68c1be1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:46:03.634762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:03.634762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290daf0a-2320-4c33-b91f-7e7ddb287e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:46:03.634762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1159/2360] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:46:03 2022] DELETE /identity/v3/projects/6385e1aaf7c94db3a3e144a68c1be1c0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:03.639477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:03.639958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:03.640077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:03.667184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:46:03.668678 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:03.726242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1202/2361] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:46:03.733958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:03.734287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:03.734574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:03.754191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:46:03.755298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:03.764720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1160/2362] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:46:03.769232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:03.779138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4b07-33e7-403c-9372-a73738c3e960 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:03.783880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:03.784548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:03.784832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:03.785107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:03.785839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:46:03.786171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:03.787820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:03.799971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1203/2363] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:46:03.804283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:03.813674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521472b6-d11b-4272-b2fb-99222ac8778c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:03.818700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:03.819347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:03.819627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:03.819897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:03.820575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:03.822474 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:03.936148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1161/2364] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:46:03 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:46:03.940394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:03.951028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9626b-ff82-44f8-9869-322973c5404f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.005136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.005802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.006087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.006396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.007095 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.009003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.114047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1204/2365] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:46:03 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:46:04.118222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.129237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74e97e-6d87-4332-9ca3-b4b08b9f3613 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.133935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.134618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.134901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.135178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.135902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.138236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.148374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1162/2366] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:46:04.152531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.162465 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f2162-eb56-4c02-8538-34c0de6fd7b8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.167569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.168454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.168454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.168803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.169470 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.171119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4679-52e5-4a3c-9b7b-977aa8ad2ea3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.181457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1205/2367] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:46:04.186615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.196497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20588010-940d-4e33-a09d-5eeac9244205 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.202042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.202964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.203252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.203528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] PATH_INFO: `/v3/projects/9e4b46ef34e5427099d165908b550df9/users/43eb61fddba5497b9443a63043a59016/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.220027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4b46ef34e5427099d165908b550df9, user_id=43eb61fddba5497b9443a63043a59016, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.222719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.305977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1163/2368] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:46:04 2022] PUT /identity/v3/projects/9e4b46ef34e5427099d165908b550df9/users/43eb61fddba5497b9443a63043a59016/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:04.318545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.320600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.320600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.346478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] MFA Rules not processed for user `43eb61fddba5497b9443a63043a59016`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:46:04.347694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:04.411260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1206/2369] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:46:04.415831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.477293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8fd75-81da-4fec-875e-bf859a563566 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.482066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.482741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.483023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.483296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.483983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.485869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.567981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1164/2370] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:46:04 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:46:04.572075 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.589302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f499d3a-4961-4fc5-bdbe-b5e80e091216 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.653557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.654331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.654625 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.654954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.655859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.657845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.754785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1207/2371] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:46:04 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:46:04.758832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.769779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2700e64-a53e-443e-9fd5-ed1e3311a720 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.774690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.775356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.775643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.775924 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.776672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.778313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.788963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1165/2372] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:46:04.794409 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.804487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2ca5-b7b9-44d4-8f95-c05552a9e198 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:04.809880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:04.810830 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:04.811112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:04.811382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] PATH_INFO: `/v3/projects/3f3da16da5ad43c0af069bcc21ecab09/users/39a53c3193de43dfbb08585898a9e8ac/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:04.826158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3da16da5ad43c0af069bcc21ecab09, user_id=39a53c3193de43dfbb08585898a9e8ac, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:04.828858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:04.941886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1208/2373] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:46:04 2022] PUT /identity/v3/projects/3f3da16da5ad43c0af069bcc21ecab09/users/39a53c3193de43dfbb08585898a9e8ac/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:04.946029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:04.956244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cfc63-9866-43d2-8a00-769d220ad72f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:05.011919 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:05.012502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:05.012787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:05.013064 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:05.013789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:05.015425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f19b4-cc8a-4674-85af-4e6cf790b406 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:05.024950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1166/2374] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:46:05.029754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:46:05.039806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ac7d6-f953-4681-a6af-67c48aa5326c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:46:05.046129 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:46:05.047036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:05.047312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:05.047584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] PATH_INFO: `/v3/projects/3f3da16da5ad43c0af069bcc21ecab09/users/39a53c3193de43dfbb08585898a9e8ac/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:05.049532 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3da16da5ad43c0af069bcc21ecab09, user_id=39a53c3193de43dfbb08585898a9e8ac, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:46:05.052188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:46:05.062899 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1209/2375] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:46:05 2022] PUT /identity/v3/projects/3f3da16da5ad43c0af069bcc21ecab09/users/39a53c3193de43dfbb08585898a9e8ac/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:46:05.073071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:46:05.073365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:46:05.073783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:46:05.102334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] MFA Rules not processed for user `39a53c3193de43dfbb08585898a9e8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:46:05.103551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:46:05.176299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1167/2376] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:47:00.778090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:47:00.780117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:47:00.838851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15bf7-1197-4f03-bdf6-f81d17a099cf admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:47:00.842539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:47:00.843196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:47:00.843478 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:47:00.843754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:47:00.845556 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:47:00.918609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:47:00.920044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:47:00.931365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1210/2377] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:47:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:47:01.548931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:47:01.558730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296c314-610b-46ad-a34b-a34380fba5a3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:47:01.562350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:47:01.562995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:47:01.563285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:47:01.563565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:47:01.566006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:47:01.579936 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:47:01.581330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:47:01.593810 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1168/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:47:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:47:04.440575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:47:04.442088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:47:04.496974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474df848-eafb-443b-8c33-16e35cb4f3cc service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:47:04.500516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:47:04.501164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:47:04.501439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:47:04.501710 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:47:04.503518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:47:04.582233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:47:04.583769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:47:04.602917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1211/2379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:47:04.699293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:47:04.708620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1530d-183b-4a7a-a0a3-ca512a6551a3 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:47:04.713154 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:47:04.713800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:47:04.714079 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:47:04.714393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:47:04.723889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:47:04.725305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:47:04.735021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1169/2380] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:48:01.726781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:01.729162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:01.730595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:01.742831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5a305-2843-46b5-ad81-84bc60f8fd82 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:01.746438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:01.747107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:01.747389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:01.747666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:01.749429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:01.789870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4e2de-edb4-433b-a108-6d6ccea41cbc service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:01.794489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:01.795358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:01.795642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:01.795917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] PATH_INFO: `/v3/users/43eb61fddba5497b9443a63043a59016` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:01.797106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43eb61fddba5497b9443a63043a59016)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:01.799070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:01.820909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:01.822395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa53c1-4d28-4b98-b7fd-5ddd4f8c1ef6 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:01.844860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1170/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:48:01.995413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43eb61fddba5497b9443a63043a59016'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:01.996172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43eb61fddba5497b9443a63043a59016'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.008454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43eb61fddba5497b9443a63043a59016'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.015936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43eb61fddba5497b9443a63043a59016'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.016779 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43eb61fddba5497b9443a63043a59016'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.092998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1212/2382] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:48:01 2022] DELETE /identity/v3/users/43eb61fddba5497b9443a63043a59016 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:02.097663 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:02.103560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:02.103560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:02.117369 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:02.118761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:02.180280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1171/2383] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:48:02.193786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:02.195226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:02.251944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb6f0b-5804-4b20-8371-24aeb132bce3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:02.256000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:02.256643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:02.256926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:02.257203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:02.266235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:02.267675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:02.277531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1213/2384] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:48:02.735465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:02.737235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:02.757998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b671b2-4bcc-4d87-a6d2-09e739630edb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:02.757998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:02.757998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:02.761437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:02.761743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] PATH_INFO: `/v3/projects/9e4b46ef34e5427099d165908b550df9` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:02.763058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4b46ef34e5427099d165908b550df9)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:02.764995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:02.881518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4b46ef34e5427099d165908b550df9', 'request_id': 'req-48bb66d6-5b37-4b7a-89b9-0960a7415f40'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.953827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4b46ef34e5427099d165908b550df9', 'request_id': 'req-48bb66d6-5b37-4b7a-89b9-0960a7415f40'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.960711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4b46ef34e5427099d165908b550df9', 'request_id': 'req-48bb66d6-5b37-4b7a-89b9-0960a7415f40'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.972558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] The token cache is being invalidate because project 9e4b46ef34e5427099d165908b550df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:48:02.972776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.973431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:02.975585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1172/2385] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:48:02 2022] DELETE /identity/v3/projects/9e4b46ef34e5427099d165908b550df9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:02.979535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:02.989868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2efe51-ce50-4e4c-814f-ddfa30349b13 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:03.045113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:03.045989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:03.046296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:03.046575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] PATH_INFO: `/v3/users/39a53c3193de43dfbb08585898a9e8ac` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:03.047800 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a53c3193de43dfbb08585898a9e8ac)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:03.049023 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:03.049712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:03.050442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:03.123862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb66d6-5b37-4b7a-89b9-0960a7415f40 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:03.129532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:03.130816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:03.131109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:03.131396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:03.133196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:03.200354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:03.201981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:03.220451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a53c3193de43dfbb08585898a9e8ac'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:03.221361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a53c3193de43dfbb08585898a9e8ac'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:03.221522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1173/2386] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:48:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:48:03.233952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a53c3193de43dfbb08585898a9e8ac'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:03.241784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a53c3193de43dfbb08585898a9e8ac'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:03.251552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a63387a1-4200-41de-a2be-3930a90437bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a53c3193de43dfbb08585898a9e8ac'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:03.350281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1214/2387] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:48:02 2022] DELETE /identity/v3/users/39a53c3193de43dfbb08585898a9e8ac => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:03.761616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:03.763081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:03.777007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76c78-dc27-49c0-8cca-06f4b5786bdd service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:03.837400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:03.838308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:03.838625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:03.838908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] PATH_INFO: `/v3/projects/3f3da16da5ad43c0af069bcc21ecab09` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:03.840104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3da16da5ad43c0af069bcc21ecab09)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:03.842054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:03.961338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3da16da5ad43c0af069bcc21ecab09', 'request_id': 'req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:04.027706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3da16da5ad43c0af069bcc21ecab09', 'request_id': 'req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:04.033730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3da16da5ad43c0af069bcc21ecab09', 'request_id': 'req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:04.045207 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] The token cache is being invalidate because project 3f3da16da5ad43c0af069bcc21ecab09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:48:04.045499 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:04.046252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9bea8059-5d2d-4e39-b3c9-ff7a7d2d9d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:48:04.052519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1174/2388] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:48:03 2022] DELETE /identity/v3/projects/3f3da16da5ad43c0af069bcc21ecab09 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:04.053698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.054163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.054299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.072178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:04.073406 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:04.130311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1215/2389] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:48:04.138129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.138458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.138731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.157028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:04.158285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:04.167024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1175/2390] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:48:04.171459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.181664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12754479-d616-4555-a4ee-8870cd3deb02 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.186224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.186886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.187165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.188157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.188157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:48:04.188667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.190151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.202519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1216/2391] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:48:04.206798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.216510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d97d3-134b-4ab0-b783-29bc8928e42d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.221084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.221727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.222006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.222317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.223003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.225987 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.355832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1176/2392] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/projects => generated 401 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:04.360043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.370682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594561b-02e0-46a4-a679-f0783f4f3981 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.428237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.428913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.429195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.429473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.430155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.432107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.515949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1217/2393] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/users => generated 301 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:04.520262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.530887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4b5b4-1d2c-4031-bccc-dfb3af5610d2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.535552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.536744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.537025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.537412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.538136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.539757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.550124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1177/2394] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:48:04.554469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.564399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2643-31c2-4fe5-afd0-e04031b5833f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.569351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.570001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.570312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.570590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.571316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.573048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1ec74-f94d-45a9-bd33-20915fd1d6d9 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.584680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1218/2395] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:48:04.588767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.599420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d748f-1b54-492f-a32b-c93e348b7e95 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.603990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.604867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.605167 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.605442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] PATH_INFO: `/v3/projects/927d51666864438c8c1dfbb4fc8144ff/users/96cfdbf8311e418abf0a798954c0b50e/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.620528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927d51666864438c8c1dfbb4fc8144ff, user_id=96cfdbf8311e418abf0a798954c0b50e, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.623222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:04.716636 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1178/2396] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:48:04 2022] PUT /identity/v3/projects/927d51666864438c8c1dfbb4fc8144ff/users/96cfdbf8311e418abf0a798954c0b50e/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:04.728687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.728985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.729271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.756050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] MFA Rules not processed for user `96cfdbf8311e418abf0a798954c0b50e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:04.757308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:04.819372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1219/2397] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:48:04.824000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:04.885722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff6df3-b5a8-4b76-adcf-6866c487ca24 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:04.890497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:04.891113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:04.891402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:04.891683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:04.892381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:04.894305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.015176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1179/2398] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:48:04 2022] POST /identity/v3/projects => generated 401 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:05.019088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:05.029901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e31d-c4b8-48d4-8006-068cfa801a3e None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:05.085232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:05.085898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.086081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.086389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.087068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:05.088966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.174740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1220/2399] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:48:05 2022] POST /identity/v3/users => generated 307 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:48:05.179522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:05.189868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416feb84-09bf-448a-9ee3-bee5e789c928 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:05.195444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:05.196103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.196386 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.196668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.197389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:05.199011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.209516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1180/2400] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:48:05.213656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:05.226576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5b2ee-b4d8-47f8-868c-26700101fe20 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:05.232048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:05.232934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.233218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.233521 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] PATH_INFO: `/v3/projects/2f865195954e4625ac18cd01c253fcde/users/e680082f734641099e775d74584c3712/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.249679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f865195954e4625ac18cd01c253fcde, user_id=e680082f734641099e775d74584c3712, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:05.252361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.350963 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1221/2401] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:48:05 2022] PUT /identity/v3/projects/2f865195954e4625ac18cd01c253fcde/users/e680082f734641099e775d74584c3712/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:48:05.354930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:05.366019 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859199a8-ba01-4d97-90ea-31bf0c1f92aa admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:05.420206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:05.420868 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.421150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.421422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.422635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:05.423795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85541b-7e89-45fb-a6dc-cc739dc4ade7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.434979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1181/2402] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:48:05.439145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:48:05.449321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa6911-98b6-4eea-8f39-dbe9c547baeb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:48:05.454267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:48:05.455146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.455423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.455694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] PATH_INFO: `/v3/projects/2f865195954e4625ac18cd01c253fcde/users/e680082f734641099e775d74584c3712/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.457697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f865195954e4625ac18cd01c253fcde, user_id=e680082f734641099e775d74584c3712, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:48:05.461080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:48:05.471792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1222/2403] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:48:05 2022] PUT /identity/v3/projects/2f865195954e4625ac18cd01c253fcde/users/e680082f734641099e775d74584c3712/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:48:05.483376 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead5794-0aa3-4385-808b-df41ed3f89b5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:48:05.483665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead5794-0aa3-4385-808b-df41ed3f89b5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:48:05.483941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead5794-0aa3-4385-808b-df41ed3f89b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:48:05.512859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-eead5794-0aa3-4385-808b-df41ed3f89b5 None None] MFA Rules not processed for user `e680082f734641099e775d74584c3712`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:48:05.514083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-eead5794-0aa3-4385-808b-df41ed3f89b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:48:05.581263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1182/2404] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:49:05.475798 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.477488 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.510970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.511531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.511705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.537534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261290-145e-44ca-bf9f-4bcda998b8a2 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.538282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] MFA Rules not processed for user `8d788b9560504c3795dbb623f10467cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:05.539493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.541449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.542143 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.542456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.542733 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.544612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.594864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1183/2405] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:49:05.628844 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:05.630480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:05.645791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.654395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1223/2406] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:05.705388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.715509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9ea9d-f1a9-454d-8722-b314d737828e service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.724481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.725202 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.725427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.725702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.726324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bb0bf-b870-4191-bbab-8cfb613876d9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.731196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.732072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.732352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.732624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] PATH_INFO: `/v3/users/96cfdbf8311e418abf0a798954c0b50e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.733846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96cfdbf8311e418abf0a798954c0b50e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:05.736647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:05.737393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:05.739079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:05.751874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1224/2407] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:49:05.915393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:05.916871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.968562 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96cfdbf8311e418abf0a798954c0b50e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:05.987514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96cfdbf8311e418abf0a798954c0b50e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:05.988496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003e553-f24d-4a0a-a52e-11f17d54e09b service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:05.988496 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:05.988731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96cfdbf8311e418abf0a798954c0b50e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:05.988851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:05.988960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96cfdbf8311e418abf0a798954c0b50e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:05.989064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:05.989064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:05.989064 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:05.995055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96cfdbf8311e418abf0a798954c0b50e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:06.004380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.006004 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99679347-e5e6-4e83-beb2-ca8d37fdbb00 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.015240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1225/2408] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:49:06.163078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1184/2409] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:49:05 2022] DELETE /identity/v3/users/96cfdbf8311e418abf0a798954c0b50e => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:49:06.167884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.168179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.168454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.187481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:06.188744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.250530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1226/2410] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:49:06.261683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.263105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.320778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34b196-b2d2-45cd-b19a-edbbb8ab359d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.324667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.325324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.325601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.325870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.335944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.337390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.347088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1185/2411] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:06.471785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.473850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.546217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee744-16d3-4bd7-a09f-2081f7c893f2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.552215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.552889 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.553173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.553453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.555289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.568492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.575327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:06.583874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1227/2412] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:06.870876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:06.878796 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:06.885692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee117d17-39c4-4e41-87ec-87dd24d31819 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:06.890749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:06.891610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:06.891963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:06.892310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] PATH_INFO: `/v3/projects/927d51666864438c8c1dfbb4fc8144ff` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:06.893575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927d51666864438c8c1dfbb4fc8144ff)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:06.895641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.083805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927d51666864438c8c1dfbb4fc8144ff', 'request_id': 'req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.176779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927d51666864438c8c1dfbb4fc8144ff', 'request_id': 'req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.183823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927d51666864438c8c1dfbb4fc8144ff', 'request_id': 'req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.196486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] The token cache is being invalidate because project 927d51666864438c8c1dfbb4fc8144ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:49:07.196772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.197421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.199113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1186/2413] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:49:06 2022] DELETE /identity/v3/projects/927d51666864438c8c1dfbb4fc8144ff => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:07.203176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:07.213981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ae7ed-d9db-479e-9175-719574e7b2ab service placement] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:07.274736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:07.275621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:07.275905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:07.276179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] PATH_INFO: `/v3/users/e680082f734641099e775d74584c3712` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:07.277382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e680082f734641099e775d74584c3712)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:07.279416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:07.612914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e680082f734641099e775d74584c3712'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.613583 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e680082f734641099e775d74584c3712'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.625575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e680082f734641099e775d74584c3712'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.632875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e680082f734641099e775d74584c3712'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.633561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fd26cc7a-1827-4cce-ae57-6f74a4d7b8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e680082f734641099e775d74584c3712'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:07.737223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1228/2414] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:49:07 2022] DELETE /identity/v3/users/e680082f734641099e775d74584c3712 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:08.250150 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:08.252788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:08.266065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab222a86-5fc5-46a2-ac78-20ef4cd49b87 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:08.326516 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:08.328068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.328352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.328626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] PATH_INFO: `/v3/projects/2f865195954e4625ac18cd01c253fcde` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.330774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f865195954e4625ac18cd01c253fcde)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:08.332705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:08.503825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f865195954e4625ac18cd01c253fcde', 'request_id': 'req-b1b2eec8-326a-4420-8b20-a915f14f2047'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:08.612097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f865195954e4625ac18cd01c253fcde', 'request_id': 'req-b1b2eec8-326a-4420-8b20-a915f14f2047'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:08.627059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f865195954e4625ac18cd01c253fcde', 'request_id': 'req-b1b2eec8-326a-4420-8b20-a915f14f2047'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:08.630559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] The token cache is being invalidate because project 2f865195954e4625ac18cd01c253fcde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:49:08.630854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:08.631515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b1b2eec8-326a-4420-8b20-a915f14f2047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:49:08.633151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1187/2415] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:49:08 2022] DELETE /identity/v3/projects/2f865195954e4625ac18cd01c253fcde => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:08.641653 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.646971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.646971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.664543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:08.665750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:08.729246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1229/2416] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:49:08.738320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.740490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.740490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.758351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:08.759484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:08.767605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1188/2417] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:49:08.772211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:08.781978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e59b-f5b9-4306-b55f-93f67977ca99 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:08.786998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:08.787660 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.787945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.788223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.788951 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:49:08.789261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:08.790919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:08.803327 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1230/2418] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:49:08.807714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:08.817795 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd317c6c-fe12-458b-ba53-d66cdf721078 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:08.824353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:08.825400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.825686 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.825966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.826701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:08.828619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:08.921304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1189/2419] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:49:08.926378 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:08.936955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e159-bd84-452d-877b-475743b6f856 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:08.995421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:08.996106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:08.996389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:08.996667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:08.997367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:08.999343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.143426 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1231/2420] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:49:08 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:49:09.147542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.158531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b28e-a6d5-4f06-951b-6d1da667ff14 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.163206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.163800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.164082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.164360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.165087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.166717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.178345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1190/2421] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:49:09.185613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.197109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d020-72e0-4503-9e5e-892817abdfb0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.205425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.205425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.205425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.205827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.206303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.208080 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb95003-0771-44b6-b709-75f5f3dcd198 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.219211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1232/2422] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:49:09.224414 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.234720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239f82-90ba-4d9d-b24c-efa2b78b866e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.240389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.241273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.241607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.241894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] PATH_INFO: `/v3/projects/5ece3dbdf3d74916bdf1ee1d410f6d35/users/92abc7227c734122b09e2824fb7546c8/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.259129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ece3dbdf3d74916bdf1ee1d410f6d35, user_id=92abc7227c734122b09e2824fb7546c8, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.261877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.373478 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1191/2423] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:49:09 2022] PUT /identity/v3/projects/5ece3dbdf3d74916bdf1ee1d410f6d35/users/92abc7227c734122b09e2824fb7546c8/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:49:09.385429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.385822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.386109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.415979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] MFA Rules not processed for user `92abc7227c734122b09e2824fb7546c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:49:09.417130 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:09.484443 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1233/2424] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:49:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:49:09.496827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:09.499634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:09.603823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35710f-2c9f-4173-8486-2bba458431e6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:09.609279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:09.618895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:09.618895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:09.618895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:09.623837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:09.626606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:09.639353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1192/2425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:10.528003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:10.529458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:10.589637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fc8ed-82d9-4cb8-b638-08a2c7aa966b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:10.593425 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:10.594074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:10.594390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:10.594668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:10.597153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:10.611537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:10.613226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:10.622079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1234/2426] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:11.568700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:11.570176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:11.662277 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e3d2-5c0b-44f4-892a-9cea9beae5f8 service glance] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:11.669742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:11.670445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:11.670731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:11.671010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:11.672837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:11.688241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:11.690668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:11.703135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1193/2427] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:49:16.196424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:49:16.197780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:16.213734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd410f-b8b2-4843-a878-ff875ba1fb52 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:49:16.219268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:49:16.220967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:49:16.221247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:49:16.221519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:49:16.223304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:49:16.239091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:49:16.248706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:49:16.254245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1235/2428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:49:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:50:56.056528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:50:56.058708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:56.185637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294844c-ab14-4c49-9516-de370356d49f service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:50:56.191039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:50:56.191791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:50:56.192191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:50:56.192567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:50:56.194590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:50:56.207006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:50:56.208555 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:50:56.218284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1194/2429] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:50:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:51:58.515507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:51:58.526224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:58.560985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7480-be8f-4b97-8c54-7d216d5af50a service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:51:58.568696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:51:58.568696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:51:58.568696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:51:58.568696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:51:58.569138 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:51:58.633820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:51:58.635310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:51:58.645358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1236/2430] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:02.748755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:02.750634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:02.769424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56da3-a8a8-4dc6-88e6-876a078a6b42 service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:02.774629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:02.775489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:02.776016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:02.776480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] PATH_INFO: `/v3/users/9dae4d7a57f343c8bb7b92dbebea181d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:02.777781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dae4d7a57f343c8bb7b92dbebea181d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:02.779767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:03.056479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dae4d7a57f343c8bb7b92dbebea181d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.056479 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dae4d7a57f343c8bb7b92dbebea181d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.068317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dae4d7a57f343c8bb7b92dbebea181d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.069043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dae4d7a57f343c8bb7b92dbebea181d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.076000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-add91fbf-f346-43d2-9e72-d7fd355f43fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dae4d7a57f343c8bb7b92dbebea181d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.155857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1195/2431] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:02 2022] DELETE /identity/v3/users/9dae4d7a57f343c8bb7b92dbebea181d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:03.454750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:03.465863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74e1eb-520c-4ab2-a149-2b4164d23def service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:03.526067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:03.526968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:03.527249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:03.527525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] PATH_INFO: `/v3/projects/8549d2b7b890411f878fe6fe776e65d7` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:03.528706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8549d2b7b890411f878fe6fe776e65d7)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:03.530716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:03.716534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8549d2b7b890411f878fe6fe776e65d7', 'request_id': 'req-175e7469-be11-4edf-9de1-5f565cedefd8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.833704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8549d2b7b890411f878fe6fe776e65d7', 'request_id': 'req-175e7469-be11-4edf-9de1-5f565cedefd8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.833704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8549d2b7b890411f878fe6fe776e65d7', 'request_id': 'req-175e7469-be11-4edf-9de1-5f565cedefd8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.841059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] The token cache is being invalidate because project 8549d2b7b890411f878fe6fe776e65d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:03.841417 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.842163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-175e7469-be11-4edf-9de1-5f565cedefd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:03.843895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1237/2432] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:03 2022] DELETE /identity/v3/projects/8549d2b7b890411f878fe6fe776e65d7 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:03.849955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa19991-d77a-4530-b766-16618891461d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:03.850952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa19991-d77a-4530-b766-16618891461d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:03.851416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa19991-d77a-4530-b766-16618891461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:03.886057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6fa19991-d77a-4530-b766-16618891461d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:03.887484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6fa19991-d77a-4530-b766-16618891461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:03.952620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1196/2433] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:03.961069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:03.961359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:03.961693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:03.980405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:03.981547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:03.989986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1238/2434] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:03.994574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa19991-d77a-4530-b766-16618891461d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.004495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa19991-d77a-4530-b766-16618891461d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.009770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.010495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.010785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.011065 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.011799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:04.012109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.013718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.026832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1197/2435] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:04.037343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.047107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f33592-630b-4502-912a-88d53e01e0e8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.051832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.052482 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.052762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.053035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.053783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.055853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.197168 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1239/2436] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:04 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:04.201284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.212252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b91e-8bb7-4501-9461-0ad9b26c9140 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.270790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.271378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.271658 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.271980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.273746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.275674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.379237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1198/2437] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:04 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:04.384012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.394546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa35e-fdee-455d-be2c-8d8cd03b7772 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.399590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.400261 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.400545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.400852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.401590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.403233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.414537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1240/2438] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:04.419037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.429583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b6ffc-f583-453e-b435-3c9854bf580c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.434220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.434954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.435237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.435515 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.436239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.437997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5d068-2322-471d-a064-e8f457585057 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.448705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1199/2439] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:04.453354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.463794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb55dc-5485-4399-b896-81881a19e817 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.468717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.469309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.469590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.469862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] PATH_INFO: `/v3/projects/4c2c44ecd51f4d73ab42fafc4750452e/users/7ed1d7dae5d343fabafc9e28473eaf65/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.486467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2c44ecd51f4d73ab42fafc4750452e, user_id=7ed1d7dae5d343fabafc9e28473eaf65, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.489153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.634949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1241/2440] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:04 2022] PUT /identity/v3/projects/4c2c44ecd51f4d73ab42fafc4750452e/users/7ed1d7dae5d343fabafc9e28473eaf65/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:04.646010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.656123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.656123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.673007 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] MFA Rules not processed for user `7ed1d7dae5d343fabafc9e28473eaf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:04.674231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:04.743593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1200/2441] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:04.754986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.757614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:04.880167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff7c-9ccf-4ca1-97e1-6d941dbe0933 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:04.887491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:04.888262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:04.888644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:04.889013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:04.903684 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:04.905540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:04.918991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1242/2442] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:04.983888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:04.985295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:05.053252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400069d1-907d-43af-b141-f87db804d8e5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:05.057130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:05.057775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:05.058054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:05.058368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:05.060111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:05.075181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:05.076582 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:05.088061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1201/2443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:05.905843 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:05.916725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eff4-4342-4507-8a63-d9d5ca8cb529 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:05.922640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:05.922640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:05.922640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:05.922640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:05.925137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:05.940706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:05.942339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:05.957405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1243/2444] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:06.255052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:06.256004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:06.314518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6707-ed94-45e4-97d0-2943ab32ec61 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:06.318171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:06.319106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:06.319911 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:06.320191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:06.321963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:06.386191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:06.387548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa8fdc-0ec4-4df8-8a14-aa257b157e47 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:06.398524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1202/2445] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:09.669046 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:09.675540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d75-0d32-495b-9211-716127a9182e service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:09.687258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:09.688956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:09.689250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:09.689566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] PATH_INFO: `/v3/users/7ed1d7dae5d343fabafc9e28473eaf65` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:09.691466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed1d7dae5d343fabafc9e28473eaf65)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:09.693539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:09.998317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed1d7dae5d343fabafc9e28473eaf65'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:09.998884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed1d7dae5d343fabafc9e28473eaf65'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.011895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed1d7dae5d343fabafc9e28473eaf65'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.019311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed1d7dae5d343fabafc9e28473eaf65'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.020101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed1d7dae5d343fabafc9e28473eaf65'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.140456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1244/2446] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:09 2022] DELETE /identity/v3/users/7ed1d7dae5d343fabafc9e28473eaf65 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:10.144319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:10.144319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:10.144319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:10.166482 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:10.168013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:10.262813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1203/2447] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:10.271568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:10.273490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:10.342609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91e69e-59b5-4d62-802b-43326b52c352 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:10.347110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:10.347794 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:10.348078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:10.348356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:10.359762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:10.361727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:10.373370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1245/2448] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:10.430814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:10.432958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:10.448108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218d33-b04d-4f1c-8459-3060e1fc636b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:10.452862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:10.453739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:10.454021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:10.454330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] PATH_INFO: `/v3/users/ecb4c97543e1452d8e6c8f1564addc76` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:10.455638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb4c97543e1452d8e6c8f1564addc76)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:10.457564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:10.735729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb4c97543e1452d8e6c8f1564addc76'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.736392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb4c97543e1452d8e6c8f1564addc76'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.750534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb4c97543e1452d8e6c8f1564addc76'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.751218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb4c97543e1452d8e6c8f1564addc76'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.758798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48f2cf66-b2cf-4aa0-99e3-84721a00db91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb4c97543e1452d8e6c8f1564addc76'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:10.849122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:10.850642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:10.864118 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a17b2-a929-4d61-8264-3dbed14b1640 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:10.869217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:10.870057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:10.870402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:10.870685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] PATH_INFO: `/v3/projects/4c2c44ecd51f4d73ab42fafc4750452e` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:10.871912 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2c44ecd51f4d73ab42fafc4750452e)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:10.873835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:10.884231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1204/2449] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:10 2022] DELETE /identity/v3/users/ecb4c97543e1452d8e6c8f1564addc76 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:11.011929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2c44ecd51f4d73ab42fafc4750452e', 'request_id': 'req-32a44189-4e9c-4245-ab8e-fa3eeabc5448'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.107883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2c44ecd51f4d73ab42fafc4750452e', 'request_id': 'req-32a44189-4e9c-4245-ab8e-fa3eeabc5448'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.114648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2c44ecd51f4d73ab42fafc4750452e', 'request_id': 'req-32a44189-4e9c-4245-ab8e-fa3eeabc5448'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.128115 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] The token cache is being invalidate because project 4c2c44ecd51f4d73ab42fafc4750452e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:11.128404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.129065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.130754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1246/2450] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:10 2022] DELETE /identity/v3/projects/4c2c44ecd51f4d73ab42fafc4750452e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:11.136631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66108bc-305d-41d4-b485-694cd762b8e6 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.136915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66108bc-305d-41d4-b485-694cd762b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.137190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66108bc-305d-41d4-b485-694cd762b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.153687 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.157222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-a66108bc-305d-41d4-b485-694cd762b8e6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:11.159578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a66108bc-305d-41d4-b485-694cd762b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.164119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.232967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1205/2451] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:11.242377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.242664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.243151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.245946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44189-4e9c-4245-ab8e-fa3eeabc5448 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.252106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.253155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.253450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.253728 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] PATH_INFO: `/v3/projects/fc0e8bf4c69e49d1b4a44905d8c9bc98` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.255109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0e8bf4c69e49d1b4a44905d8c9bc98)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.257783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.267331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:11.269367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.285303 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1206/2452] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:11.292080 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.315142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aaf06d-767f-4281-aa66-64ec1f7bf766 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.325054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.325998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.326589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.327058 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.328458 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:11.328989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.331848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.354681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1207/2453] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:11.359192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.370536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc4330-14d9-42db-8c3b-847112d21f59 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.375362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.376059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.376394 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.376678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.377407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.379364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.453605 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0e8bf4c69e49d1b4a44905d8c9bc98', 'request_id': 'req-944e231d-3129-41d1-b071-12db474e745b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.544196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0e8bf4c69e49d1b4a44905d8c9bc98', 'request_id': 'req-944e231d-3129-41d1-b071-12db474e745b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.551731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0e8bf4c69e49d1b4a44905d8c9bc98', 'request_id': 'req-944e231d-3129-41d1-b071-12db474e745b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.556590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1208/2454] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:11.558331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.567441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] The token cache is being invalidate because project fc0e8bf4c69e49d1b4a44905d8c9bc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:11.567741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.568474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-944e231d-3129-41d1-b071-12db474e745b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:11.572925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1247/2455] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:11 2022] DELETE /identity/v3/projects/fc0e8bf4c69e49d1b4a44905d8c9bc98 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:11.579743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48891-28bf-4c48-a16b-8fb511c960b3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.586011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48891-28bf-4c48-a16b-8fb511c960b3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.586011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d48891-28bf-4c48-a16b-8fb511c960b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.599588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b2d48891-28bf-4c48-a16b-8fb511c960b3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:11.601789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b2d48891-28bf-4c48-a16b-8fb511c960b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.632237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87222d17-b8b9-42e6-a2c8-31556b27ba0d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.641203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.642210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.642500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.642953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.643821 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.645957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.673117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1248/2456] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:11.680976 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.689310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.689310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.703339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:11.704971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.716530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1249/2457] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:11.721177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.731967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5ff06-070e-4a7e-8723-b2ff4c454c68 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.737116 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.737850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.738225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.738504 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.739234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:11.739542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.741797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.756845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1250/2458] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:52:11.762225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.772522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1209/2459] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:11.784028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.784028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:11.784256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5db1f-2c6e-4220-830c-39112a71e91c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.796822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.797897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.798428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.798888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.806673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.809057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.824669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f377f-f00a-4676-90df-a13a67ff7ae4 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.844536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.845876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.846174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.846502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.847390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.851118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.862227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1210/2460] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:11.865823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.876717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fbb1e-2024-4a81-b5c7-543a2eed4266 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.885297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.886094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.886312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.886595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.887328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.889117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.901088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1211/2461] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:11.905202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:11.915627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7719d8d-3d20-464f-8cf3-0cc8490e5c2c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:11.920472 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:11.921338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:11.921623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:11.921897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] PATH_INFO: `/v3/projects/df74131cba184b5095adf00962093262/users/d4ebfa9c169e4b7eb011aa93f1405781/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:11.938133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df74131cba184b5095adf00962093262, user_id=d4ebfa9c169e4b7eb011aa93f1405781, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:11.941010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d35dff-eb6c-4982-b5da-119e67d81cbb admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:11.968498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1251/2462] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:11.972652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.039194 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a145-85fd-4596-a48d-f9e3c262c142 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.046821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.047644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.047935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.048205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.048894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.051296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.088761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1212/2463] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:11 2022] PUT /identity/v3/projects/df74131cba184b5095adf00962093262/users/d4ebfa9c169e4b7eb011aa93f1405781/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:12.099096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.099390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.099775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.128614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] MFA Rules not processed for user `d4ebfa9c169e4b7eb011aa93f1405781`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:12.130389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:12.160752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1252/2464] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:11 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:12.165475 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.175943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a2469-53e1-40eb-b073-d136538817a4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.199556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1213/2465] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:12.261263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.261263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.261263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.261263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.261744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.261744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.261744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1253/2466] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:12.264120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.264321 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.274668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b748d5-c022-4a71-9a01-e80ec975a60f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.276337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bde698-b8d5-4045-84e2-06269cb6b6bd None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.279453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.280051 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.280334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.280734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.281429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.281641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.282398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.282745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.283086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.283363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.283980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.285620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.296578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1214/2467] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:12.300992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.312743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b85-eb51-49d0-8279-8c8a0dc8efff admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.317900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.319985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.320262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.320536 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] PATH_INFO: `/v3/projects/6f2d3eeb73834766abf555de78e7a23a/users/1f792f2111a247bea71ed65ae2a61a6a/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.343060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2d3eeb73834766abf555de78e7a23a, user_id=1f792f2111a247bea71ed65ae2a61a6a, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.346117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf28e54-0cb0-4a4a-8289-4cdf4adcf451 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.400589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1254/2468] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:12 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:12.404318 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.422417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1215/2469] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:12 2022] PUT /identity/v3/projects/6f2d3eeb73834766abf555de78e7a23a/users/1f792f2111a247bea71ed65ae2a61a6a/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:12.433033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.433342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.433611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.466421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] MFA Rules not processed for user `1f792f2111a247bea71ed65ae2a61a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:12.467999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:12.486419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1b4fb-f4a4-42fc-bd5c-8f63a0082c6d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.559382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1216/2470] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:12.565260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.566805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:12.570783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.571265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.571623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.571909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.572607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.574840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.649214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2e3db-dc96-46ae-bada-0f89ce260651 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.654950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.655602 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.655886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.656166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.673954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.673954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.677662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1217/2471] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:12.701588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1255/2472] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:12 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:12.706964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.735243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a342-5479-4c28-80ba-607317af580f service nova] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.737816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.738554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.738838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.739117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.739840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.741545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.754300 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1218/2473] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:12.757446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.768275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71565d3-c3db-4ab6-a794-1671ad268ae4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.779100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.779100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.779100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.779100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] PATH_INFO: `/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581/users/cdf57c0648c44129b345128969fa1ff5/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.795564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c56d8a5b22e4c6cb74f218543c2d581, user_id=cdf57c0648c44129b345128969fa1ff5, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.798561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:12.919406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.924091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1256/2474] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:12 2022] PUT /identity/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581/users/cdf57c0648c44129b345128969fa1ff5/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:12.927911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:12.930632 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcbcb3-4fc3-48c3-b75f-4d4d32a58815 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:12.994096 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:12.995034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:12.995320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:12.995599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] PATH_INFO: `/v3/users/1f792f2111a247bea71ed65ae2a61a6a` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:12.996854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f792f2111a247bea71ed65ae2a61a6a)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:12.998867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.005263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85245c-26f8-4077-97d5-62413e6ade03 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:13.012066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:13.016783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.016783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.016783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.016783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:13.016783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.026927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1257/2475] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:13.031258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:13.041974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dff870-b375-490c-8180-0060efd0f73c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:13.047065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:13.047942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.048222 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.048494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] PATH_INFO: `/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581/users/cdf57c0648c44129b345128969fa1ff5/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.050522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c56d8a5b22e4c6cb74f218543c2d581, user_id=cdf57c0648c44129b345128969fa1ff5, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:13.053846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba7451-9b6b-48f6-9b14-3d2cda69fa96 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.064871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1258/2476] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:13 2022] PUT /identity/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581/users/cdf57c0648c44129b345128969fa1ff5/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:13.077267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.077558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.077839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.105662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] MFA Rules not processed for user `cdf57c0648c44129b345128969fa1ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:13.106926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.176292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1259/2477] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:13.195184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:13.196730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.224212 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f792f2111a247bea71ed65ae2a61a6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:13.225731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f792f2111a247bea71ed65ae2a61a6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:13.251784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f792f2111a247bea71ed65ae2a61a6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:13.251784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f792f2111a247bea71ed65ae2a61a6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:13.251784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f792f2111a247bea71ed65ae2a61a6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:13.258621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bbc83-4c53-4f18-a793-99b769a7e4e5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:13.262372 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:13.263027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.263312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.263588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.265391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.278919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:13.280390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02bc86-d693-4af6-bc3d-fc493c3aa593 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.290957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1260/2478] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:13.362490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1219/2479] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:12 2022] DELETE /identity/v3/users/1f792f2111a247bea71ed65ae2a61a6a => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:13.367924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.368210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.368486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.388648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:13.389279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.463487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1261/2480] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:13.473654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:13.475606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.518464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:13.520131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.541844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca5ff9-7187-4198-a7e1-e9ed65c05409 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:13.545697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:13.546093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.546511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.546712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.556788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:13.558425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.569230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1220/2481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:13.587180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7f614-1ae8-47eb-9f3f-bff123e3f281 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:13.590466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:13.591224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:13.591512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:13.591784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:13.593599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:13.657991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:13.659469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e861a-21ff-4bd8-ad31-f6fa1bc20701 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:13.669026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1262/2482] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:14.009572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.011011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:14.024687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c67cb5-6794-4cb5-92ae-bd55b1d8be18 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.029715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.030830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.031115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.031574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] PATH_INFO: `/v3/projects/6f2d3eeb73834766abf555de78e7a23a` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.032942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2d3eeb73834766abf555de78e7a23a)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.035029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.167407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2d3eeb73834766abf555de78e7a23a', 'request_id': 'req-4899ffa0-fcad-4b48-9da4-dad77096cfad'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:14.252580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2d3eeb73834766abf555de78e7a23a', 'request_id': 'req-4899ffa0-fcad-4b48-9da4-dad77096cfad'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:14.259178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2d3eeb73834766abf555de78e7a23a', 'request_id': 'req-4899ffa0-fcad-4b48-9da4-dad77096cfad'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:14.270580 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] The token cache is being invalidate because project 6f2d3eeb73834766abf555de78e7a23a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:14.270875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:14.272131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4899ffa0-fcad-4b48-9da4-dad77096cfad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:14.273720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1221/2483] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:14 2022] DELETE /identity/v3/projects/6f2d3eeb73834766abf555de78e7a23a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:14.278813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.279896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.279896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.297774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:14.299070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:14.365088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1263/2484] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:14.372973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.373262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.373538 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.392654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:14.393854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:14.403054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1222/2485] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:14.407302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.417913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f293170-c3d5-4042-8fdb-65836cab91e0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.422560 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.423217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.423520 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.423799 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.424522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:14.424821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.426473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.438382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1264/2486] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:14.442871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.452669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50bacf-e122-4f46-ae54-aaae3be76aa3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.458048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.458760 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.459588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.460018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.460721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.462866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.574750 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1223/2487] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:14 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:14.579534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.590911 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d390959-556c-464a-92f5-bbb8a4bfa358 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.648813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.649492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.649778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.650060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.650780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.652693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.791288 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1265/2488] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:14 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:14.796395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.807439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2516-c6aa-4043-ab38-ea7796a049ef admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.812157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.812824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.813104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.813378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.814129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.815787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.826722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1224/2489] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:14.831101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.843469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb731a2-8d89-48b5-8377-2b03a88068f7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.848255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.848917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.849203 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.850016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.850801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.852396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe764ecb-a1ad-4239-94f2-4331aaa409b2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:14.862820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1266/2490] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:14.868383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:14.878740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89693718-df26-4683-a9ca-1cbec5a2e865 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:14.883464 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:14.884354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:14.884639 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:14.884918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] PATH_INFO: `/v3/projects/f4109778936543e895b4a0a5251d0f3d/users/11fab63761a04958a4252b1c96798a01/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:14.902290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4109778936543e895b4a0a5251d0f3d, user_id=11fab63761a04958a4252b1c96798a01, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:14.904959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.008106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1225/2491] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:14 2022] PUT /identity/v3/projects/f4109778936543e895b4a0a5251d0f3d/users/11fab63761a04958a4252b1c96798a01/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:15.019904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.020199 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.020483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.048774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] MFA Rules not processed for user `11fab63761a04958a4252b1c96798a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:15.049995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:15.122023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1267/2492] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:15.126744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.210415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d4a90-0826-484a-a4c7-136d36b24872 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.215109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.215771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.216054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.216330 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.217046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.218984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.327655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1226/2493] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:15 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:15.328835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.340617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2be9-c723-48ca-80f3-1bb7bddc79ac None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.395732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.397056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.397334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.397612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.398344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.400276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.505018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1268/2494] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:15 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:15.510188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.522876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c2c6a-963e-45bb-9c9b-eb3f815a9906 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.527493 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.528336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.528749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.531185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.531185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.531471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.544854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1227/2495] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:15.549716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.560513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681dda7f-a2b0-436d-8839-53933fe69ca4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.565163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.566041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.566354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.566635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] PATH_INFO: `/v3/projects/798d1f468095462785738a411999f3f3/users/fbd31b11ffb145c796de13b1a821b3b4/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.581763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798d1f468095462785738a411999f3f3, user_id=fbd31b11ffb145c796de13b1a821b3b4, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.585235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.673782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1269/2496] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:15 2022] PUT /identity/v3/projects/798d1f468095462785738a411999f3f3/users/fbd31b11ffb145c796de13b1a821b3b4/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:15.678310 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.688651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53167-7caa-4c6c-8bd6-1e5dc1a2c09f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.745295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.745977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.746332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.746615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.747586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.748980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc06198-916a-43c0-8222-7bb286739375 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.759481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1228/2497] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:15.763566 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.774083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f668-5084-4814-b643-4ab6c1b27e94 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:15.778662 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:15.779534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.779813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.780082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] PATH_INFO: `/v3/projects/798d1f468095462785738a411999f3f3/users/fbd31b11ffb145c796de13b1a821b3b4/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.782034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798d1f468095462785738a411999f3f3, user_id=fbd31b11ffb145c796de13b1a821b3b4, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:15.784703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:15.795043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1270/2498] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:15 2022] PUT /identity/v3/projects/798d1f468095462785738a411999f3f3/users/fbd31b11ffb145c796de13b1a821b3b4/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:15.805768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:15.806087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:15.808270 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:15.842255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] MFA Rules not processed for user `fbd31b11ffb145c796de13b1a821b3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:15.843465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:15.920999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1229/2499] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:15.933063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:15.934815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:16.000543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06683a4a-275c-4ae1-8811-d87ab9381d13 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:16.005481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:16.007274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:16.007555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:16.007828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:16.019785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:16.021195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:16.033117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1271/2500] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:16.324926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:16.337021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557acb0-98e4-493d-a56d-a51f96101b8c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:16.341069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:16.341705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:16.341983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:16.342288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:16.413089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:16.414614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64789a-eb2a-4b2d-beee-ff452b1ebfa1 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:16.424672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1230/2501] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:32.877326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:32.878974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:32.946176 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390189-f3c2-498e-822c-17496d265bd8 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:32.950827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:32.951714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:32.951988 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:32.952266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] PATH_INFO: `/v3/users/92abc7227c734122b09e2824fb7546c8` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:32.953519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92abc7227c734122b09e2824fb7546c8)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:32.955507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:33.256235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92abc7227c734122b09e2824fb7546c8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:33.256902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92abc7227c734122b09e2824fb7546c8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:33.269196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92abc7227c734122b09e2824fb7546c8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:33.278250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92abc7227c734122b09e2824fb7546c8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:33.278888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92abc7227c734122b09e2824fb7546c8'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:33.377665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1272/2502] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:32 2022] DELETE /identity/v3/users/92abc7227c734122b09e2824fb7546c8 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:33.383585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:33.384118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:33.384118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:33.403136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:33.404350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:33.468556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1231/2503] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:33.478742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:33.480193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:33.542983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb36a13-6447-4e82-91fb-8588728a9523 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:33.546941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:33.547569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:33.547860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:33.548135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:33.558586 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:33.560033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701c747-7163-4d7c-9418-79040fdf0287 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:33.571226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1273/2504] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:34.048125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:34.050113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:34.079984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8f741-1de1-43e6-bfef-836b1ed4a9ae None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:34.089751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:34.091385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.091867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.092151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] PATH_INFO: `/v3/projects/5ece3dbdf3d74916bdf1ee1d410f6d35` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.093382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ece3dbdf3d74916bdf1ee1d410f6d35)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:34.096120 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:34.259060 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ece3dbdf3d74916bdf1ee1d410f6d35', 'request_id': 'req-28f1bdc5-ce63-452e-8cfc-72c9587cba26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:34.361427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ece3dbdf3d74916bdf1ee1d410f6d35', 'request_id': 'req-28f1bdc5-ce63-452e-8cfc-72c9587cba26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:34.370171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ece3dbdf3d74916bdf1ee1d410f6d35', 'request_id': 'req-28f1bdc5-ce63-452e-8cfc-72c9587cba26'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:34.382251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] The token cache is being invalidate because project 5ece3dbdf3d74916bdf1ee1d410f6d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:34.383716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:34.383716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-28f1bdc5-ce63-452e-8cfc-72c9587cba26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:34.384890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1232/2505] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:34 2022] DELETE /identity/v3/projects/5ece3dbdf3d74916bdf1ee1d410f6d35 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:34.394895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.396467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.396467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.415503 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:34.416839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:34.488961 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1274/2506] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:34.495460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.495781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.496169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.516948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:34.519456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:34.526992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1233/2507] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:34.532111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:34.543230 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073619e-897d-4474-9a59-5df32f1633fd None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:34.548201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:34.559756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:34.566613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1275/2508] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:34.571351 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:34.581477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a05b-3ec1-481f-b9d4-0a9f8933b3e5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:34.586897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:34.587564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.587842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.588171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.588818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:34.590742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:34.760967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1234/2509] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:34 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:34.766058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:34.778070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94401e-e7b4-4098-b93f-b076de183767 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:34.851014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:34.851777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:34.852090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:34.852380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:34.853157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:34.855164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:34.981823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1276/2510] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:34 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:34.985961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:34.996436 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c645716-ec5d-49b6-a62a-de228a80d3b8 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:35.000985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:35.001646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:35.001927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:35.002256 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:35.003040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:35.005237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:35.015641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1235/2511] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:35.019629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:35.030517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ae691-cdf8-4460-bf67-015f1292e839 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:35.035212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:35.035995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:35.036404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:35.036680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:35.037401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:35.039133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4753d0-6114-4252-840b-e1325ae4e687 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:35.048872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1277/2512] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:35.058927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:35.069692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4046fa-4cf1-47b3-af8c-811bb8df4fb9 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:35.074317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:35.075246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:35.075532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:35.075812 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] PATH_INFO: `/v3/projects/bda1f4ab03184482881e7a31ae1d7854/users/a67605538b654ec4910eff76d297a992/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:35.091704 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda1f4ab03184482881e7a31ae1d7854, user_id=a67605538b654ec4910eff76d297a992, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:35.094463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:35.193494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1236/2513] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:35 2022] PUT /identity/v3/projects/bda1f4ab03184482881e7a31ae1d7854/users/a67605538b654ec4910eff76d297a992/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:35.260126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:35.260456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:35.261056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:35.291232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] MFA Rules not processed for user `a67605538b654ec4910eff76d297a992`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:35.294296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:35.365031 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1278/2514] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:35.376598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:35.378338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:35.437010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f57b0f-d678-4386-b1d3-5fd18556449f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:35.441017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee88945608d42c0981bb93470e832fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:35.441679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:35.441960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:35.442522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:35.450909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:35.452356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208093d3-6b61-44f1-bd0f-bd97e55489e2 service glance] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:35.461967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1237/2515] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:39.437273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:39.438754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:39.511974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f40f3-9e73-4027-bd97-9783647c2f21 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:39.517739 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:39.518920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:39.519205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:39.519483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] PATH_INFO: `/v3/users/11fab63761a04958a4252b1c96798a01` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:39.521627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fab63761a04958a4252b1c96798a01)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:39.523645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:39.844048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fab63761a04958a4252b1c96798a01'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:39.844835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fab63761a04958a4252b1c96798a01'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:39.857634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fab63761a04958a4252b1c96798a01'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:39.865704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fab63761a04958a4252b1c96798a01'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:39.866540 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fab63761a04958a4252b1c96798a01'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:39.981108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1279/2516] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:39 2022] DELETE /identity/v3/users/11fab63761a04958a4252b1c96798a01 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:39.999999 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.000553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.000553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.021331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:40.022193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:40.091790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1238/2517] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:40.102448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:40.103994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:40.180682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b7d74-2c67-4a14-83ba-707fd2ed71bb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:40.184929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:40.185713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.185999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.186311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.201003 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:40.202599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:40.225727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1280/2518] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:40.712917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:40.714498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:40.728440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d70f7-d388-4f1f-a60b-fbc0f9aefaab None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:40.738711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:40.739578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:40.739799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:40.740274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] PATH_INFO: `/v3/projects/f4109778936543e895b4a0a5251d0f3d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:40.741666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4109778936543e895b4a0a5251d0f3d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:40.744006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:40.893473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4109778936543e895b4a0a5251d0f3d', 'request_id': 'req-3324b9f5-482e-4ad2-973c-8bd82416843b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:40.998243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4109778936543e895b4a0a5251d0f3d', 'request_id': 'req-3324b9f5-482e-4ad2-973c-8bd82416843b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.004869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4109778936543e895b4a0a5251d0f3d', 'request_id': 'req-3324b9f5-482e-4ad2-973c-8bd82416843b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.016275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] The token cache is being invalidate because project f4109778936543e895b4a0a5251d0f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:41.016565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.017250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.018838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1239/2519] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:40 2022] DELETE /identity/v3/projects/f4109778936543e895b4a0a5251d0f3d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:41.024107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:41.034442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027243d-e3f3-4a44-9910-3198169216d5 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:41.090959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:41.091987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:41.092270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:41.092679 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] PATH_INFO: `/v3/users/fbd31b11ffb145c796de13b1a821b3b4` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:41.094056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd31b11ffb145c796de13b1a821b3b4)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:41.096460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:41.353572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd31b11ffb145c796de13b1a821b3b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.359919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd31b11ffb145c796de13b1a821b3b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.368991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd31b11ffb145c796de13b1a821b3b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.377232 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd31b11ffb145c796de13b1a821b3b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.377952 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-974803ba-1339-4957-82ba-4a2f407fb7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd31b11ffb145c796de13b1a821b3b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:41.452893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1281/2520] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:41 2022] DELETE /identity/v3/users/fbd31b11ffb145c796de13b1a821b3b4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:42.001837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.003408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:42.018165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324b9f5-482e-4ad2-973c-8bd82416843b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.080990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.081886 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.082296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.082584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] PATH_INFO: `/v3/projects/798d1f468095462785738a411999f3f3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.083981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798d1f468095462785738a411999f3f3)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.085926 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.234966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798d1f468095462785738a411999f3f3', 'request_id': 'req-ad5858ed-8330-45b0-b637-aa2b00ef5105'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:42.329696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798d1f468095462785738a411999f3f3', 'request_id': 'req-ad5858ed-8330-45b0-b637-aa2b00ef5105'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:42.337343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798d1f468095462785738a411999f3f3', 'request_id': 'req-ad5858ed-8330-45b0-b637-aa2b00ef5105'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:42.350258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] The token cache is being invalidate because project 798d1f468095462785738a411999f3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:42.350715 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:42.351495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ad5858ed-8330-45b0-b637-aa2b00ef5105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:42.353381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1240/2521] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:42 2022] DELETE /identity/v3/projects/798d1f468095462785738a411999f3f3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:42.358749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.359756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.359756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.378271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:42.379516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:42.440172 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1282/2522] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:42.450259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.450566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.450848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.469492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:42.470783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:42.479196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1241/2523] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:42.483856 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.493386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751961b-3f3b-47de-91fb-67c01bdd8cb8 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.498423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.499077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.499358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.499630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.500353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:42.500657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.502276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.516039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1283/2524] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:42.521625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.530961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b727064-1636-4acd-94cd-39fe13f2154b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.535948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.536644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.536932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.537209 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.537920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.539876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.653977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1242/2525] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:42 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:42.658824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.671492 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a606-f3e4-49d4-a14b-8b4eebc7229e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.728642 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.729344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.729623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.729913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.730611 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.732522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.842082 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1284/2526] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:42 2022] POST /identity/v3/users => generated 282 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:42.846492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.858180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaba68f-2813-41e0-97f8-c5e6466c5744 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.863059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.863887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.864168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.864431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.865156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.866789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.878052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1243/2527] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:42.882483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.892689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442635f-57c0-4dfe-9af6-4ffe6c291a4f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.898079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.898785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.899068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.899344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.900081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.901921 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5e53f-dd76-494c-a215-28b11e7c25e5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:42.913499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1285/2528] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:42.917955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:42.928413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35166414-d78a-4383-8b03-5fe8b6c84f1f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:42.933318 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:42.934221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:42.934508 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:42.934778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] PATH_INFO: `/v3/projects/753f243fce9f427dbab54b509a9951e1/users/1e354927b9a743148d4d006f78a4e04c/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:42.949524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=753f243fce9f427dbab54b509a9951e1, user_id=1e354927b9a743148d4d006f78a4e04c, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:42.952786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:43.050897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1244/2529] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:42 2022] PUT /identity/v3/projects/753f243fce9f427dbab54b509a9951e1/users/1e354927b9a743148d4d006f78a4e04c/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:43.062441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.062736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.063011 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.095220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] MFA Rules not processed for user `1e354927b9a743148d4d006f78a4e04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:43.096069 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:43.166783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1286/2530] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:43.175467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:43.237435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39e411-2685-40ea-ad11-2fdb2c3b9375 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:43.242378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:43.243099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.243510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.243713 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.244415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:43.246349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:43.344983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1245/2531] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:43 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:43.347008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:43.359032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5aa1b-3fa9-4054-989e-28e54c9d576f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:43.419575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:43.420424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.420866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.421216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.421982 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:43.424270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:43.605970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1287/2532] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:43 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:43.610522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:43.623050 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44f46b-4079-4d4e-8a5b-0d64d6124596 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:43.631332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:43.632102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.632461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.632853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.633644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:43.635347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:43.661858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1246/2533] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:43.662286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:43.672676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ce47-1ede-415d-86a7-d14c72bbf9c6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:43.678384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:43.679312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:43.679598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:43.679872 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] PATH_INFO: `/v3/projects/708fd0485d3e459fa37ead0504aef724/users/361990bdc6b1487eab4de81aa1229d4d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:43.697153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708fd0485d3e459fa37ead0504aef724, user_id=361990bdc6b1487eab4de81aa1229d4d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:43.700447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:43.953431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1288/2534] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:43 2022] PUT /identity/v3/projects/708fd0485d3e459fa37ead0504aef724/users/361990bdc6b1487eab4de81aa1229d4d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:43.957607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:43.967627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681623cc-2ad2-4adf-a8ca-378ff56bb09a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.027876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.028546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.028831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.029108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.029830 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.031456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.042352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1247/2535] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:44.046601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.059781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d05645-6943-487e-8d17-26e4c17822a8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.065676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.069681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.069681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.069681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] PATH_INFO: `/v3/projects/708fd0485d3e459fa37ead0504aef724/users/361990bdc6b1487eab4de81aa1229d4d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.070792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708fd0485d3e459fa37ead0504aef724, user_id=361990bdc6b1487eab4de81aa1229d4d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.072013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.073971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:44.074691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e6d4-60a0-47fa-b905-22e37675efe1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.086050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1289/2536] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:44 2022] PUT /identity/v3/projects/708fd0485d3e459fa37ead0504aef724/users/361990bdc6b1487eab4de81aa1229d4d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:44.098893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.098893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.098893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.135770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] MFA Rules not processed for user `361990bdc6b1487eab4de81aa1229d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:44.137131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:44.174297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a6f02-5cc5-4b93-94da-6c910c0f3b7c admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.230055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1290/2537] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:44.248149 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.255852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:44.264334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.267116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.267409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.273890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.278435 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:44.349365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859d3cc-12d8-4487-9ac8-dd4a24316167 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.357043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.358364 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.358813 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.359108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.369219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.370793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.378286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.383607 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.414039 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1248/2538] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:44.414308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1291/2539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:44.637815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:44.650542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765d7bd-39d5-4eed-8a7e-0eb5deed23e4 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:44.657119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:44.658876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:44.658876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:44.658876 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:44.758556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:44.760208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:44.771163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1249/2540] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:45.330607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:45.332148 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:45.362366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a3070-3258-4669-81e2-844bd9a330d2 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:45.369379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:45.370415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:45.370780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:45.371127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] PATH_INFO: `/v3/users/48efa16cb9844fcd853f1e3760619a66` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:45.372429 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48efa16cb9844fcd853f1e3760619a66)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:45.374498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:45.614341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48efa16cb9844fcd853f1e3760619a66'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:45.615122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48efa16cb9844fcd853f1e3760619a66'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:45.628554 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48efa16cb9844fcd853f1e3760619a66'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:45.636650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48efa16cb9844fcd853f1e3760619a66'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:45.637545 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48efa16cb9844fcd853f1e3760619a66'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:45.830311 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1292/2541] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:45 2022] DELETE /identity/v3/users/48efa16cb9844fcd853f1e3760619a66 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:46.119501 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:46.134897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814bcfd-1b7d-4004-8b93-3d41dedae15c service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:46.197951 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:46.198938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:46.199346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:46.199695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] PATH_INFO: `/v3/projects/709b9712f09a42a785d046e0c0f3846e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:46.201033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709b9712f09a42a785d046e0c0f3846e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:46.203331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:46.329175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709b9712f09a42a785d046e0c0f3846e', 'request_id': 'req-16e9827b-cf77-4e05-9dd1-b13dce2483d3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:46.432262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709b9712f09a42a785d046e0c0f3846e', 'request_id': 'req-16e9827b-cf77-4e05-9dd1-b13dce2483d3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:46.438527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709b9712f09a42a785d046e0c0f3846e', 'request_id': 'req-16e9827b-cf77-4e05-9dd1-b13dce2483d3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:46.451784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] The token cache is being invalidate because project 709b9712f09a42a785d046e0c0f3846e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:46.452116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:46.452787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-16e9827b-cf77-4e05-9dd1-b13dce2483d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:46.454629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1250/2542] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:46 2022] DELETE /identity/v3/projects/709b9712f09a42a785d046e0c0f3846e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:48.722781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:48.724244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:48.737466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e497-44e9-4921-b17c-8a14a0ec9273 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:48.797017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:48.797893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:48.798287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:48.798564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] PATH_INFO: `/v3/users/1e354927b9a743148d4d006f78a4e04c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:48.799750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e354927b9a743148d4d006f78a4e04c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:48.801959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.009673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e354927b9a743148d4d006f78a4e04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:49.010474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e354927b9a743148d4d006f78a4e04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:49.023857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e354927b9a743148d4d006f78a4e04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:49.031211 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e354927b9a743148d4d006f78a4e04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:49.031903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e354927b9a743148d4d006f78a4e04c'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:49.136768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1293/2543] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:48 2022] DELETE /identity/v3/users/1e354927b9a743148d4d006f78a4e04c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:49.141218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.141722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.141826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.159134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:49.160401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:49.219578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1251/2544] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:49.229631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.231058 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:49.297724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644375f-13cf-4a29-9888-cf50a40a87dd admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.301957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.302637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.302922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.303195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.312822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.314278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.324588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1294/2545] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:49.819488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:49.820975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:49.855305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa27fc-e691-47fb-aa01-25d6515f0677 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:49.855305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:49.855305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:49.855305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:49.856172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] PATH_INFO: `/v3/projects/753f243fce9f427dbab54b509a9951e1` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:49.856172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=753f243fce9f427dbab54b509a9951e1)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:49.856172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:49.983393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753f243fce9f427dbab54b509a9951e1', 'request_id': 'req-4c88c233-70e3-4db7-82c8-f299cb721f29'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.085193 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753f243fce9f427dbab54b509a9951e1', 'request_id': 'req-4c88c233-70e3-4db7-82c8-f299cb721f29'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.091722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753f243fce9f427dbab54b509a9951e1', 'request_id': 'req-4c88c233-70e3-4db7-82c8-f299cb721f29'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.103735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] The token cache is being invalidate because project 753f243fce9f427dbab54b509a9951e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:50.104030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.104688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.106271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1252/2546] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:49 2022] DELETE /identity/v3/projects/753f243fce9f427dbab54b509a9951e1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:50.110548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:50.120718 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6cbf4-21c7-4253-a1b2-64cd95c29ad9 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:50.177413 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:50.178476 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:50.178754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:50.179026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] PATH_INFO: `/v3/users/361990bdc6b1487eab4de81aa1229d4d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:50.180434 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361990bdc6b1487eab4de81aa1229d4d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:50.182688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:50.470914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361990bdc6b1487eab4de81aa1229d4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.471721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361990bdc6b1487eab4de81aa1229d4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.483808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361990bdc6b1487eab4de81aa1229d4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.491353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361990bdc6b1487eab4de81aa1229d4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.492038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-c366431e-bbb8-4f0e-90bd-c290034cd047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361990bdc6b1487eab4de81aa1229d4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:50.681644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1295/2547] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:50 2022] DELETE /identity/v3/users/361990bdc6b1487eab4de81aa1229d4d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:51.131694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.139324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:51.148481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c233-70e3-4db7-82c8-f299cb721f29 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:51.209770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:51.210578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.210859 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.211131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] PATH_INFO: `/v3/projects/708fd0485d3e459fa37ead0504aef724` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.212305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708fd0485d3e459fa37ead0504aef724)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:51.214282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:51.360577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708fd0485d3e459fa37ead0504aef724', 'request_id': 'req-deea362a-69b0-482c-b2e3-fac0c7eadb48'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:51.489848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708fd0485d3e459fa37ead0504aef724', 'request_id': 'req-deea362a-69b0-482c-b2e3-fac0c7eadb48'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:51.495710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708fd0485d3e459fa37ead0504aef724', 'request_id': 'req-deea362a-69b0-482c-b2e3-fac0c7eadb48'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:51.506798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] The token cache is being invalidate because project 708fd0485d3e459fa37ead0504aef724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:51.507092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:51.507753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-deea362a-69b0-482c-b2e3-fac0c7eadb48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:51.509463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1253/2548] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:51 2022] DELETE /identity/v3/projects/708fd0485d3e459fa37ead0504aef724 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:51.514541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.515175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.515283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.534167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:51.535950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:51.594277 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1296/2549] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:51.603044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.603334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.603613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.622470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:51.623672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:51.633569 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1254/2550] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:52:51.637879 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.648643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe38f1-b3f2-4a9a-a052-185c8a7ad7f3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:51.653923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:51.654626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.654941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.655217 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.655955 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:51.656259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:51.657870 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:51.671706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1297/2551] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:51.675542 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.685222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25167d-740b-45ca-8aa2-ade8e81267cc None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:51.690285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:51.691604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.692087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.692430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.693268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:51.695465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:51.796877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1255/2552] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:51 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:51.801415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.812321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4493-2089-4752-b0ab-06edba837698 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:51.872857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:51.873652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.874110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.874421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.874546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.875125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:51.876389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:51.877132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:51.934247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886a8bd-bacf-4a18-8c1d-1354af8c4161 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:51.938260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:51.943503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:51.943503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:51.943503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:51.943503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:51.954566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:51.956002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:51.966762 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1256/2553] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:52:51.981233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1298/2554] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:51 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:51.985467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:51.996251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276e0d-6529-44a7-bfd7-082ea6816889 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.002066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.002564 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.002847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.003127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.004403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.009004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.016782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1257/2555] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:52.022145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.034650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9653d-9756-491f-b48d-20afd7a0e664 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.039759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.040435 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.040755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.041035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.041771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.043410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07368a-3d06-4d7e-84fd-be112d4623d2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.056190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1299/2556] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:52.060918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.072730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76aa2b-1380-4493-b711-3d7848e10591 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.078015 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.079213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.079498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.079768 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] PATH_INFO: `/v3/projects/396703f2af0c43bf85afaf43309f2aa5/users/09f9014972a041fab4cd5fb3541fc4ba/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.115798 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=396703f2af0c43bf85afaf43309f2aa5, user_id=09f9014972a041fab4cd5fb3541fc4ba, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.118685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.251591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1258/2557] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:52 2022] PUT /identity/v3/projects/396703f2af0c43bf85afaf43309f2aa5/users/09f9014972a041fab4cd5fb3541fc4ba/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:52:52.262375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.262670 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.262945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.290769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] MFA Rules not processed for user `09f9014972a041fab4cd5fb3541fc4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:52.291932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:52.364525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1300/2558] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:52.368069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.436964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62cd53-e777-49df-aba3-526e015cbb3a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.442743 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.443405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.443689 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.443965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.444665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.446602 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.607331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1259/2559] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:52 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:52.611965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.623416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52781f26-865b-429f-a0ab-c87a321b4ae7 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.680524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.681338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.681623 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.681894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.682619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.684630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.760580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1301/2560] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:52 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:52.765116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.777214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ab157-70b0-4287-8dab-966466c96faa admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.782554 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.783306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.783813 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.784338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.785529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.787217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.799665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1260/2561] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:52.806902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.818164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397288d-280c-483a-a8f7-dccf1a4c3dea admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:52.822771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:52.823682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:52.823965 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:52.824247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] PATH_INFO: `/v3/projects/72a7f2371e2149c2b744e9e7590550ba/users/8a5b7be0974f4860bec18417fe230602/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:52.839619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a7f2371e2149c2b744e9e7590550ba, user_id=8a5b7be0974f4860bec18417fe230602, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:52.843408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:52.954764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1302/2562] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:52 2022] PUT /identity/v3/projects/72a7f2371e2149c2b744e9e7590550ba/users/8a5b7be0974f4860bec18417fe230602/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:52.954957 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:52.970228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c341ff-8981-4f63-96e4-39d0eac27998 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.036620 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.037343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.037624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.037944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.038828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.040461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692659c-4e7c-406b-b748-4412101da988 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.053573 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1261/2563] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:52:53.059196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.074034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e1b70-af08-42a5-b359-5d171ab72f6c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.079347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.080449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.080737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.081017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] PATH_INFO: `/v3/projects/72a7f2371e2149c2b744e9e7590550ba/users/8a5b7be0974f4860bec18417fe230602/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.085418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a7f2371e2149c2b744e9e7590550ba, user_id=8a5b7be0974f4860bec18417fe230602, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.089268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.114427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1303/2564] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:53 2022] PUT /identity/v3/projects/72a7f2371e2149c2b744e9e7590550ba/users/8a5b7be0974f4860bec18417fe230602/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:53.124837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.125594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.125958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.165366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] MFA Rules not processed for user `8a5b7be0974f4860bec18417fe230602`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:53.166612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:53.239281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1262/2565] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:53.251414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:53.253107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:53.311491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbfa48-7b71-4c87-95c4-29f01ab13b06 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:53.315574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:53.316925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:53.317213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:53.317490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:53.375388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:53.376848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:53.387098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1304/2566] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:52:56.030010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:56.033730 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:56.118838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b53590-3686-4acf-8a7b-2528c60e44e1 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:56.123203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:56.124082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:56.124357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:56.124630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] PATH_INFO: `/v3/users/8d788b9560504c3795dbb623f10467cd` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:56.125804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d788b9560504c3795dbb623f10467cd)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:56.127746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:56.344280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d788b9560504c3795dbb623f10467cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:56.344913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d788b9560504c3795dbb623f10467cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:56.356925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d788b9560504c3795dbb623f10467cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:56.357598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d788b9560504c3795dbb623f10467cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:56.365837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-e614af14-f90b-42dd-84f8-dfcd943773d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d788b9560504c3795dbb623f10467cd'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:56.449328 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1263/2567] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:52:56 2022] DELETE /identity/v3/users/8d788b9560504c3795dbb623f10467cd => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:56.755835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:56.768931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3255922-4bf9-46a7-8e1c-13820de85a45 service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:56.826354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:56.827247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:56.827529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:56.827802 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] PATH_INFO: `/v3/projects/2a315adb7b214dceb03ed0aac50d2519` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:56.829173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a315adb7b214dceb03ed0aac50d2519)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:56.831122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:56.942690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a315adb7b214dceb03ed0aac50d2519', 'request_id': 'req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:57.027843 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a315adb7b214dceb03ed0aac50d2519', 'request_id': 'req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:57.033572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a315adb7b214dceb03ed0aac50d2519', 'request_id': 'req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:57.045206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] The token cache is being invalidate because project 2a315adb7b214dceb03ed0aac50d2519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:52:57.045490 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:57.046134 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a24d2cac-ff9d-41ee-80a2-fc4d08a02b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:52:57.047724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1305/2568] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:52:56 2022] DELETE /identity/v3/projects/2a315adb7b214dceb03ed0aac50d2519 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:57.052661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.053171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.053288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.070939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:57.072126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:57.133503 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1264/2569] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:57.142040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.142361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.142664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.162096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:57.163312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:57.171178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1306/2570] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:57.175539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.185127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59ab3-a345-4db4-843e-3faf7084687a None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.189641 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.190349 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.190661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.190904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.191617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:52:57.191920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.193494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.204522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1265/2571] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:52:57.209236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.218431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163fee-124f-44e8-a779-dd99f33b1cad None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.224702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.225356 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.225666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.225945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.226806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.228709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.355860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1307/2572] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:57.360109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.370147 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2bdbd-4363-4e47-ad16-cbd67a03be9f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.424261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.424928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.425257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.425534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.426250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.428161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.679489 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1266/2573] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/users => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:57.683544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.694354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96458a-403a-47a7-b41d-c1524486446e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.698920 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.701818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.701818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.701818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.701818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.702533 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.714781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1308/2574] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:57.718894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.729234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6485-8648-4e17-843a-7f47b06f884e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.733972 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.734645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.734927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.735199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.735917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.737997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1200cb2-52f1-4a0c-9985-2dff7d366b36 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.748356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1267/2575] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:57.752433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:57.763369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614998-f69d-4ad7-89fc-655831face8e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:57.768215 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:57.769275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.769509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.769674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] PATH_INFO: `/v3/projects/f2454f0ec16842feaf9b3c4ab5e4637e/users/00c4365db8494279bc58de51d4b5ec56/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.783943 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2454f0ec16842feaf9b3c4ab5e4637e, user_id=00c4365db8494279bc58de51d4b5ec56, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:57.786712 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:57.889014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1309/2576] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:57 2022] PUT /identity/v3/projects/f2454f0ec16842feaf9b3c4ab5e4637e/users/00c4365db8494279bc58de51d4b5ec56/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:57.899897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:57.900189 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:57.900466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:57.927665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] MFA Rules not processed for user `00c4365db8494279bc58de51d4b5ec56`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:57.928844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:57.992934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1268/2577] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:57.997354 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.056839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81709e03-011b-4585-baa7-d9f93d2e4eb6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.061391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.062042 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.062352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.062626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.063297 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.065173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.200875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1310/2578] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:52:57 2022] POST /identity/v3/projects => generated 405 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:52:58.205329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.216112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182fa72-3c95-4330-aa86-f6f29e4bf3c9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.269677 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.270287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.270567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.270839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.271514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.273399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.382904 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1269/2579] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:52:58 2022] POST /identity/v3/users => generated 309 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:52:58.386968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.398265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774972b-2eea-4182-a48a-dd0cd31ae819 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.403016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.403686 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.403967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.404244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.405049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.406745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.418396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1311/2580] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:58.422873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.433818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b38b5-39a5-49c9-8b15-b53e03fc23f6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.438457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.439362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.439640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.439914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] PATH_INFO: `/v3/projects/1b60fd013c314f5d95400899ad917053/users/cfae43d22c8d49c6abea1945ec5971b9/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.459066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b60fd013c314f5d95400899ad917053, user_id=cfae43d22c8d49c6abea1945ec5971b9, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.461803 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.594156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1270/2581] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:58 2022] PUT /identity/v3/projects/1b60fd013c314f5d95400899ad917053/users/cfae43d22c8d49c6abea1945ec5971b9/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:58.598700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.608874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b31df-171c-4e5a-9ea8-524ac310a77c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.668537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.669241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.669525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.669805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.670591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.672192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920653c2-d4f9-47f2-9fa6-f8ef77012303 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.683486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1312/2582] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:52:58.687954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.698993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a99c1-fad6-4ac5-8769-08c6c91c3dc4 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.703740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.704629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.704905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.705178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] PATH_INFO: `/v3/projects/1b60fd013c314f5d95400899ad917053/users/cfae43d22c8d49c6abea1945ec5971b9/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.708222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b60fd013c314f5d95400899ad917053, user_id=cfae43d22c8d49c6abea1945ec5971b9, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.710890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.721710 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1271/2583] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:52:58 2022] PUT /identity/v3/projects/1b60fd013c314f5d95400899ad917053/users/cfae43d22c8d49c6abea1945ec5971b9/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:52:58.733228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.733523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.733810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.760399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] MFA Rules not processed for user `cfae43d22c8d49c6abea1945ec5971b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:52:58.761615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:58.831784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1313/2584] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:52:58.843071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:52:58.844615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:52:58.902337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac35553-677e-4d3c-afd9-e1603659a3f7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:52:58.905937 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:52:58.906599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:52:58.906880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:52:58.907157 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:52:58.915894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:52:58.917306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000293c4-f29d-406c-a225-f62dd43075ec service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:52:58.926598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1272/2585] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:01.431446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:01.498123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f928b1-e304-404b-8c7a-5b58bbb55112 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:01.502629 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:01.504097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:01.504358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:01.504705 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] PATH_INFO: `/v3/users/00c4365db8494279bc58de51d4b5ec56` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:01.505937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c4365db8494279bc58de51d4b5ec56)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:01.507917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:01.739170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c4365db8494279bc58de51d4b5ec56'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:01.739917 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c4365db8494279bc58de51d4b5ec56'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:01.752065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c4365db8494279bc58de51d4b5ec56'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:01.759748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c4365db8494279bc58de51d4b5ec56'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:01.767866 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c4365db8494279bc58de51d4b5ec56'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:01.864192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1314/2586] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:01 2022] DELETE /identity/v3/users/00c4365db8494279bc58de51d4b5ec56 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:01.869338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:01.869860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:01.869977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:01.889550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:01.890701 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:01.954543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1273/2587] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:53:01.966854 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:01.968335 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:02.052506 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ef34b-2188-4893-827b-c10322b793eb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:02.057432 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:02.058151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.058510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.059098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.071068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:02.072732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:02.084174 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1315/2588] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:02.716191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:02.726560 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:02.745644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0feed7-9678-4af7-9870-4cc03acfb3e2 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:02.752769 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:02.762325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:02.762532 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:02.764407 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] PATH_INFO: `/v3/projects/f2454f0ec16842feaf9b3c4ab5e4637e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:02.765823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2454f0ec16842feaf9b3c4ab5e4637e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:02.767778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:02.874092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2454f0ec16842feaf9b3c4ab5e4637e', 'request_id': 'req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:02.942722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2454f0ec16842feaf9b3c4ab5e4637e', 'request_id': 'req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:02.948682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2454f0ec16842feaf9b3c4ab5e4637e', 'request_id': 'req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:02.960252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] The token cache is being invalidate because project f2454f0ec16842feaf9b3c4ab5e4637e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:02.960548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:02.961197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:02.962794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1274/2589] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:02 2022] DELETE /identity/v3/projects/f2454f0ec16842feaf9b3c4ab5e4637e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:02.966805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:02.977666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc813-aec5-40be-8546-1592fc6eebbe service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.033295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.034234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.034526 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.034797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] PATH_INFO: `/v3/users/cfae43d22c8d49c6abea1945ec5971b9` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.036032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfae43d22c8d49c6abea1945ec5971b9)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.038237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.199654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfae43d22c8d49c6abea1945ec5971b9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.200296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfae43d22c8d49c6abea1945ec5971b9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.214122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfae43d22c8d49c6abea1945ec5971b9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.221186 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfae43d22c8d49c6abea1945ec5971b9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.222454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-8155fded-c347-400b-92fe-bef522b12956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfae43d22c8d49c6abea1945ec5971b9'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.290245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1316/2590] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:02 2022] DELETE /identity/v3/users/cfae43d22c8d49c6abea1945ec5971b9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:03.740054 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:03.741473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:03.757052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7cc6-536d-49c0-b766-c6909fb8e7c4 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:03.820160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:03.821024 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:03.821307 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:03.821575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] PATH_INFO: `/v3/projects/1b60fd013c314f5d95400899ad917053` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:03.822789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b60fd013c314f5d95400899ad917053)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:03.824685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:03.926982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b60fd013c314f5d95400899ad917053', 'request_id': 'req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:03.993672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b60fd013c314f5d95400899ad917053', 'request_id': 'req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:04.005192 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b60fd013c314f5d95400899ad917053', 'request_id': 'req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:04.023208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] The token cache is being invalidate because project 1b60fd013c314f5d95400899ad917053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:04.023617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:04.024678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-544fb6d0-f27b-4bf5-b9d8-da467f2e416e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:04.026527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1275/2591] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:03 2022] DELETE /identity/v3/projects/1b60fd013c314f5d95400899ad917053 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:53:04.033575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.033575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.033575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.068878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:04.071449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:04.170333 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1317/2592] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:53:04.179175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.179465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.179758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.199549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:04.200735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:04.208722 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1276/2593] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:04.213077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.223451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9aaeb-4f9d-4b49-83d7-2756bf79e2ed None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.228271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.228934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.229220 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.229499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.230255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:04.230573 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.232166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.245867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1318/2594] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:53:04.250118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.259844 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f81ec-9768-4c85-b0fa-032dbf8831fb None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.264624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.265268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.265551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.265843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.266610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.268546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.389182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1277/2595] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:53:04.393430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.403348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a868-9830-46bd-9ff1-687c62975469 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.460930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.461617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.461901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.462180 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.462914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.464858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.640637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1319/2596] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:04.644862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.660236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373caa24-5c19-4e89-a69e-c098c6cc8c3d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.664966 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.665638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.665928 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.666233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.666976 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.668627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.679846 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1278/2597] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:04.684457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.695308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85875c-d78f-428b-b3a0-dbe375c1fa3c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.700073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.700778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.701093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.701367 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.702112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.703751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5945-b801-4aad-853a-9679028c43f4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.714726 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1320/2598] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:53:04.719612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:04.731736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea315-c876-4a98-bb20-33f0c61e5301 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:04.736737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:04.738712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.739178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.739467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] PATH_INFO: `/v3/projects/8d45f39eb67047f2876c9bfd5c24e3ee/users/4f7cb21b23f9447780ee72c203cf842c/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.756997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d45f39eb67047f2876c9bfd5c24e3ee, user_id=4f7cb21b23f9447780ee72c203cf842c, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:04.759873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:04.845238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1279/2599] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:04 2022] PUT /identity/v3/projects/8d45f39eb67047f2876c9bfd5c24e3ee/users/4f7cb21b23f9447780ee72c203cf842c/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:04.857206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef684-8688-416d-8247-1b0ef439213a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:04.857507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef684-8688-416d-8247-1b0ef439213a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:04.858047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef684-8688-416d-8247-1b0ef439213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:04.888888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-442ef684-8688-416d-8247-1b0ef439213a None None] MFA Rules not processed for user `4f7cb21b23f9447780ee72c203cf842c`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:04.890187 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-442ef684-8688-416d-8247-1b0ef439213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:04.957198 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1321/2600] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:04.961839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.023540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d59be2-5f6c-4f49-8cb9-11c7f07d6c22 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.028162 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.028820 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.029100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.029372 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.030066 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.031986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.159826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1280/2601] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:04 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:05.164184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef684-8688-416d-8247-1b0ef439213a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.176547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef684-8688-416d-8247-1b0ef439213a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.232034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.232985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.233265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.233538 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.234279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.236190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.334755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1322/2602] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:05 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:05.338847 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.350456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627d36-0966-4075-9c36-e0a0c5d6e528 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.355099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.355747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.356056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.361194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.361194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.361194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.370661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1281/2603] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:05.374654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.384584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732ff0-50bd-4571-804a-de3e92c0e025 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.392280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.393201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.393486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.393753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] PATH_INFO: `/v3/projects/6079fef1c405460d9b418f8332f64c1d/users/3ca00274210a436f80aaa7163451493d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.412658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6079fef1c405460d9b418f8332f64c1d, user_id=3ca00274210a436f80aaa7163451493d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.415371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.506106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1323/2604] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:05 2022] PUT /identity/v3/projects/6079fef1c405460d9b418f8332f64c1d/users/3ca00274210a436f80aaa7163451493d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:05.510046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.519986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab97-7c7d-4b87-bc79-01609ed97918 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.578550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.579230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.579519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.579799 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.580559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.582317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587c9c7-b124-4f50-823d-6de4f9d6ff61 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.592234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1282/2605] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:05.597021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.608767 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91dca9-0485-4f03-ac7f-4dfa10e238a6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.615183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.616078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.616358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.616632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] PATH_INFO: `/v3/projects/6079fef1c405460d9b418f8332f64c1d/users/3ca00274210a436f80aaa7163451493d/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.618851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6079fef1c405460d9b418f8332f64c1d, user_id=3ca00274210a436f80aaa7163451493d, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.621700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.632818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1324/2606] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:05 2022] PUT /identity/v3/projects/6079fef1c405460d9b418f8332f64c1d/users/3ca00274210a436f80aaa7163451493d/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:05.643776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.644161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.644609 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.672585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] MFA Rules not processed for user `3ca00274210a436f80aaa7163451493d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:05.673786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:05.740811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1283/2607] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:05.752059 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:05.754332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:05.810928 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71671-4104-4178-92e6-94272517e438 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:05.814550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:05.815205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:05.815487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:05.815765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:05.824770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:05.826316 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147004-7043-4185-a93b-b49cb6747f0e service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:05.836513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1325/2608] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:06.863630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:06.927426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25474-8887-4c13-9064-5d1b1514d39b None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:06.933392 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:06.938343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:06.938343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:06.938343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] PATH_INFO: `/v3/users/4f7cb21b23f9447780ee72c203cf842c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:06.938343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7cb21b23f9447780ee72c203cf842c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:06.938343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.241669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7cb21b23f9447780ee72c203cf842c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:07.242380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7cb21b23f9447780ee72c203cf842c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:07.255008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7cb21b23f9447780ee72c203cf842c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:07.256272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7cb21b23f9447780ee72c203cf842c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:07.266504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7cb21b23f9447780ee72c203cf842c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:07.370941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1284/2609] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:06 2022] DELETE /identity/v3/users/4f7cb21b23f9447780ee72c203cf842c => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:07.375457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.375985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.376120 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.394873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:07.396189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:07.459827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1326/2610] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:07.471087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:07.471427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:07.533078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2608-e6f7-4379-a413-5fdb3ef4b69d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:07.536895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:07.537550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:07.537848 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:07.538122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:07.548470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:07.550022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:07.561388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1285/2611] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:08.018667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:08.020105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:08.033839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6cec-9692-4a01-beab-674606cbe624 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:08.039869 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:08.040639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:08.040948 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:08.041191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] PATH_INFO: `/v3/projects/8d45f39eb67047f2876c9bfd5c24e3ee` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:08.042480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d45f39eb67047f2876c9bfd5c24e3ee)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:08.044593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.152246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d45f39eb67047f2876c9bfd5c24e3ee', 'request_id': 'req-072e49ae-2cab-4709-b063-4cd3977f5692'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.233285 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d45f39eb67047f2876c9bfd5c24e3ee', 'request_id': 'req-072e49ae-2cab-4709-b063-4cd3977f5692'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.240218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d45f39eb67047f2876c9bfd5c24e3ee', 'request_id': 'req-072e49ae-2cab-4709-b063-4cd3977f5692'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.251276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] The token cache is being invalidate because project 8d45f39eb67047f2876c9bfd5c24e3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:08.251567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.252219 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.253913 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1327/2612] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:08 2022] DELETE /identity/v3/projects/8d45f39eb67047f2876c9bfd5c24e3ee => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:08.257913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:08.267858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9af54-743d-4469-a94c-6c5e1c4fc143 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:08.326002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:08.326791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:08.327076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:08.327356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] PATH_INFO: `/v3/users/3ca00274210a436f80aaa7163451493d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:08.328574 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca00274210a436f80aaa7163451493d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:08.331160 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:08.500430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca00274210a436f80aaa7163451493d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.501222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca00274210a436f80aaa7163451493d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.515172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca00274210a436f80aaa7163451493d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.515913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca00274210a436f80aaa7163451493d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.524576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-152bb3c7-6733-45d0-86cd-1fbe46cab60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca00274210a436f80aaa7163451493d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:08.623731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1286/2613] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:08 2022] DELETE /identity/v3/users/3ca00274210a436f80aaa7163451493d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:53:09.295123 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:09.296770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:09.316877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e49ae-2cab-4709-b063-4cd3977f5692 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:09.375505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:09.376363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.376640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.376907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] PATH_INFO: `/v3/projects/6079fef1c405460d9b418f8332f64c1d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.378105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6079fef1c405460d9b418f8332f64c1d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:09.380052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:09.492947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6079fef1c405460d9b418f8332f64c1d', 'request_id': 'req-052a9e16-4506-4a39-925f-0da51af09d9f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:09.582571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6079fef1c405460d9b418f8332f64c1d', 'request_id': 'req-052a9e16-4506-4a39-925f-0da51af09d9f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:09.589289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6079fef1c405460d9b418f8332f64c1d', 'request_id': 'req-052a9e16-4506-4a39-925f-0da51af09d9f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:09.604231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] The token cache is being invalidate because project 6079fef1c405460d9b418f8332f64c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:09.604509 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:09.605181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-052a9e16-4506-4a39-925f-0da51af09d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:09.606778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1328/2614] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:09 2022] DELETE /identity/v3/projects/6079fef1c405460d9b418f8332f64c1d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:09.611920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.612512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.612887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.631566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:09.632745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:09.692633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1287/2615] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:09.702701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.703070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.703352 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.723390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:09.724571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:09.732824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1329/2616] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:09.737045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:09.746739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801737f-fee3-45a8-a37f-f54d6c9a1a52 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:09.752356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:09.753873 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.754158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.754465 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.755305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:09.755615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:09.757506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:09.770417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1288/2617] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:09.779238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:09.788735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f4c62-074a-4081-9ea0-e5230a3c9453 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:09.793848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:09.794544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.794831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.795106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.795804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:09.797725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:09.898620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1330/2618] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:09 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:53:09.902790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:09.913005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea89a9-7f02-4f35-a93e-f78159e81c58 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:09.967592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:09.968259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:09.968543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:09.968818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:09.969496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:09.971703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.054494 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1289/2619] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:09 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:10.058584 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.069874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7510-0d9c-49ab-a480-402a983a849d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.078284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.078284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.078284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.078284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.079240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.079240 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.089457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1331/2620] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:10.093631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.108199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac2b4-66b8-4a87-8af5-9f91847e29a2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.108566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.109220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.109504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.109786 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.110543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.112654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830801d-d16d-4fcc-8feb-22f5e199d445 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.122669 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1290/2621] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:53:10.126776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.137355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c070e-6b35-4ad1-8f7c-4cb2f3c8e170 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.141985 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.142888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.143437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.143721 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] PATH_INFO: `/v3/projects/527b4a6f297842e5a6c8ea87c1ca34d3/users/c2f44444154b43a3be027677afcc4b93/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.157764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=527b4a6f297842e5a6c8ea87c1ca34d3, user_id=c2f44444154b43a3be027677afcc4b93, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.160561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.256774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1332/2622] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:10 2022] PUT /identity/v3/projects/527b4a6f297842e5a6c8ea87c1ca34d3/users/c2f44444154b43a3be027677afcc4b93/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:10.268091 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.268381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.268653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.294263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] MFA Rules not processed for user `c2f44444154b43a3be027677afcc4b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:10.295455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:10.358705 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1291/2623] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:10.363790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.423181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969bab3-c0c4-4de2-884d-d3098707d742 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.428494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.429154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.429438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.429744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.430468 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.432370 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.531181 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1333/2624] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:10 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:53:10.535540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.546329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4535b14-ab28-4a57-a0cd-cb8735a85677 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.601296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.601963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.602272 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.602553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.603229 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.605132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.713217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1292/2625] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:10 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:10.717119 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.728393 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91653c1-696b-4e39-bc00-734c94f04159 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.733493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.734153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.734593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.734880 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.735601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.737893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.748518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1334/2626] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:10.752699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.763148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fb65-2e8c-4e6d-9efb-14f62df46e9b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.767667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.769695 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.769982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.770295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] PATH_INFO: `/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef/users/b75179e481ed4d27979c97a77582a470/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.785253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b268bb688ad45f18b5b1fc2ef589aef, user_id=b75179e481ed4d27979c97a77582a470, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.787970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.863211 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1293/2627] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:10 2022] PUT /identity/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef/users/b75179e481ed4d27979c97a77582a470/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:10.866969 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.876934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26b8a3-952a-4b00-ac44-9effd2dd7c5d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.936047 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.936723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.937008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.937286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.938015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.939643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5d894-c658-42fb-85b6-e690c39f4414 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.949846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1335/2628] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:10.954026 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:10.965366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f7d9-b1ee-421f-9580-1ecc8c48a83a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:10.969897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:10.970818 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.971199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.972139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] PATH_INFO: `/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef/users/b75179e481ed4d27979c97a77582a470/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:10.974425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b268bb688ad45f18b5b1fc2ef589aef, user_id=b75179e481ed4d27979c97a77582a470, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:10.977205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:10.988044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1294/2629] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:10 2022] PUT /identity/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef/users/b75179e481ed4d27979c97a77582a470/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:10.998685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:10.999015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:10.999259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.025574 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] MFA Rules not processed for user `b75179e481ed4d27979c97a77582a470`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:11.026824 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:11.094746 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1336/2630] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:11.106365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:11.108329 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:11.168112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accacc64-8ee0-4784-89fd-5a264560109a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:11.172035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:11.173897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:11.173897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:11.173897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.188477 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:11.189991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbfb90-102a-4517-8efa-46a683f464be service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:11.200398 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1295/2631] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:11.338735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:11.400486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388974-cd06-424a-92ee-76954d0dff90 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:11.405541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:11.412365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:11.412365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:11.412365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] PATH_INFO: `/v3/users/c2f44444154b43a3be027677afcc4b93` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.412365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f44444154b43a3be027677afcc4b93)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:11.412365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:11.668757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f44444154b43a3be027677afcc4b93'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:11.669763 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f44444154b43a3be027677afcc4b93'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:11.689724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f44444154b43a3be027677afcc4b93'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:11.700109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f44444154b43a3be027677afcc4b93'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:11.700842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f44444154b43a3be027677afcc4b93'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:11.794142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1337/2632] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:11 2022] DELETE /identity/v3/users/c2f44444154b43a3be027677afcc4b93 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:11.796863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:11.797529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:11.797853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:11.829744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:11.832280 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:11.932901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1296/2633] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:11.944536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:11.946555 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:12.072959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d913ca2-d5bb-4a67-a676-0846f05c9307 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:12.080091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:12.081000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:12.085094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:12.085094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:12.105376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:12.108338 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:12.122760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1338/2634] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:12.583634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:12.585592 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:12.600758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c4bc9-90d9-4a4d-b27c-cd171c97116d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:12.604751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:12.605914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:12.606420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:12.606703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] PATH_INFO: `/v3/projects/527b4a6f297842e5a6c8ea87c1ca34d3` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:12.608243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527b4a6f297842e5a6c8ea87c1ca34d3)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:12.610279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:12.727645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527b4a6f297842e5a6c8ea87c1ca34d3', 'request_id': 'req-5f32e564-5482-4436-8388-2afb46ad04f1'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:12.787238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527b4a6f297842e5a6c8ea87c1ca34d3', 'request_id': 'req-5f32e564-5482-4436-8388-2afb46ad04f1'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:12.793244 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527b4a6f297842e5a6c8ea87c1ca34d3', 'request_id': 'req-5f32e564-5482-4436-8388-2afb46ad04f1'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:12.805793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] The token cache is being invalidate because project 527b4a6f297842e5a6c8ea87c1ca34d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:12.806558 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:12.806764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:12.808396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1297/2635] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:12 2022] DELETE /identity/v3/projects/527b4a6f297842e5a6c8ea87c1ca34d3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:12.813091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:12.822954 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79d6f-6be8-4fec-b0c4-8fba56a2b8f7 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:12.882454 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:12.883332 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:12.883613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:12.883887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] PATH_INFO: `/v3/users/b75179e481ed4d27979c97a77582a470` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:12.885136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75179e481ed4d27979c97a77582a470)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:12.887093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:13.065321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75179e481ed4d27979c97a77582a470'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.066055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75179e481ed4d27979c97a77582a470'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.077814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75179e481ed4d27979c97a77582a470'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.084957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75179e481ed4d27979c97a77582a470'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.085689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-37cb2deb-89ad-47e3-b04f-01e57937f135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75179e481ed4d27979c97a77582a470'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.165048 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1339/2636] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:53:12 2022] DELETE /identity/v3/users/b75179e481ed4d27979c97a77582a470 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:13.605874 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:13.607412 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:13.624216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e564-5482-4436-8388-2afb46ad04f1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:13.684552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:13.685408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:13.685688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:13.685962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] PATH_INFO: `/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:13.687182 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b268bb688ad45f18b5b1fc2ef589aef)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:13.689133 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:13.798590 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b268bb688ad45f18b5b1fc2ef589aef', 'request_id': 'req-4747b2cf-6296-4311-b171-d9e9a9de0d9b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.919056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b268bb688ad45f18b5b1fc2ef589aef', 'request_id': 'req-4747b2cf-6296-4311-b171-d9e9a9de0d9b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.926018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b268bb688ad45f18b5b1fc2ef589aef', 'request_id': 'req-4747b2cf-6296-4311-b171-d9e9a9de0d9b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.937992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] The token cache is being invalidate because project 7b268bb688ad45f18b5b1fc2ef589aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:53:13.938317 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.939035 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-4747b2cf-6296-4311-b171-d9e9a9de0d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:53:13.941417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1298/2637] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:53:13 2022] DELETE /identity/v3/projects/7b268bb688ad45f18b5b1fc2ef589aef => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:13.946474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:13.946968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:13.947088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.006630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:14.007840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:14.068057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1340/2638] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:14.076842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.077125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.077401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.098986 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:14.100203 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:14.108699 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1299/2639] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:14.113501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.123361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcdd9-05ef-4e78-a8f8-63e850455716 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.129235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.129953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.130271 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.130558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.131299 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:53:14.131608 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.133292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.146846 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1341/2640] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:53:14.151342 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.161692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5acd-8d0f-4ad8-b581-2cce7261ac26 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.166733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.167506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.167681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.168754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.168754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.170633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.321862 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1300/2641] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:14 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:14.326017 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.336459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f43a-4518-4eab-abe9-7630cd413fef admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.396612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.397291 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.397604 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.397892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.398620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.400552 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.495061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1342/2642] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:14 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:14.499761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.512664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c44af-2be5-44f9-b3f0-909111c1e8da admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.518674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.519557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.519840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.520114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.520841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.523185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.534021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1301/2643] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:53:14.538419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.548396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36d9a-ff5a-4644-a0a9-752a82c8b3b8 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.553837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.554541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.554823 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.555379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.556127 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.557731 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16f2d0-0865-4582-ba89-4f5a384b24e0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.568342 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1343/2644] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:14.572759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.583228 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a240e71-1c42-4a85-bbc4-65bdb577e7dc admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.588010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.588894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.589172 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.589446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] PATH_INFO: `/v3/projects/fe86a9041e0042b68458b93a6fde5006/users/ca85cb037d0b44bfb5cc24f9a9bc463b/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.604884 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe86a9041e0042b68458b93a6fde5006, user_id=ca85cb037d0b44bfb5cc24f9a9bc463b, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.607954 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:14.688735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1302/2645] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:14 2022] PUT /identity/v3/projects/fe86a9041e0042b68458b93a6fde5006/users/ca85cb037d0b44bfb5cc24f9a9bc463b/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:14.750416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.750704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.750981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.778827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] MFA Rules not processed for user `ca85cb037d0b44bfb5cc24f9a9bc463b`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:14.779999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:14.845709 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1344/2646] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:53:14.851195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:14.913664 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29debebf-f61c-4c66-a80f-849bb5138ae7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:14.920751 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:14.921408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:14.921693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:14.921997 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:14.922716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:14.924652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.029101 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1303/2647] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:53:14 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:15.033246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.045400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bd566-ac10-492a-a802-59b17cf2d213 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.115295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.115991 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.116280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.116558 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.117382 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.119341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.279632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1345/2648] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:53:15 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:53:15.284175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.294872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2a319-6937-49dc-9974-aed45d4e5e0b admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.299568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.300242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.300706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.300977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.301714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.303346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.315298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1304/2649] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:15.320188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.331270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5129060-2a4a-4a33-b79a-c13bb75284ec admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.337225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.338135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.338451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.338735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] PATH_INFO: `/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673/users/592c345d2d484dfb95b310cd8f70e165/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.358420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85bb63bb7c14d549b3daaf9ece2b673, user_id=592c345d2d484dfb95b310cd8f70e165, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.361210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.445369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1346/2650] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:15 2022] PUT /identity/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673/users/592c345d2d484dfb95b310cd8f70e165/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:15.449381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.460969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c74-67c5-4224-a29b-86cc4ce76df2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.520754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.521553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.521840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.522121 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.522900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.524514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2297a0e-4d08-4913-b2dd-49f57242bfe2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.537996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1305/2651] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:53:15.546379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.557914 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0b98-f2b3-4d02-9ffc-e82ffa478107 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.562793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.563685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.563968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.564248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] PATH_INFO: `/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673/users/592c345d2d484dfb95b310cd8f70e165/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.567388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85bb63bb7c14d549b3daaf9ece2b673, user_id=592c345d2d484dfb95b310cd8f70e165, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.571438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.582243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1347/2652] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:53:15 2022] PUT /identity/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673/users/592c345d2d484dfb95b310cd8f70e165/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:53:15.592995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.593332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.593640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.627095 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] MFA Rules not processed for user `592c345d2d484dfb95b310cd8f70e165`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:53:15.628856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:15.712797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1306/2653] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:53:15.724265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.724509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:15.725983 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:15.726154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:15.788076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe824c9-0ede-40bd-b58e-f2f1d0c6d6d4 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.789101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60401a4-1480-4a35-93c6-c7e616065158 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:15.792077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.792739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.792867 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:15.793050 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.793295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.793466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:15.793755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:15.794040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:15.796369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:53:15.875854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.877753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.880167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:15.882614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:15.888286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1307/2654] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:53:15.900789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1348/2655] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:53:37.217914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:53:37.227916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a03d3-4963-4fe3-8479-c49bb1efdc95 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:53:37.233333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:53:37.233333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:53:37.233333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:53:37.233333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:53:37.244225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:53:37.245671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:53:37.257013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1308/2656] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:53:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:03.626808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:03.700544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae369ec-b385-4a6b-9750-3bd74a220b6b service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:03.705633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:03.706559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:03.706835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:03.707113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] PATH_INFO: `/v3/users/ca85cb037d0b44bfb5cc24f9a9bc463b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:03.708359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca85cb037d0b44bfb5cc24f9a9bc463b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:03.710518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:03.848565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:03.850359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:03.865628 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aba3-54e2-4246-8a6e-538951469b84 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:03.870522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:03.871396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:03.871833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:03.871960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] PATH_INFO: `/v3/users/09f9014972a041fab4cd5fb3541fc4ba` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:03.873220 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f9014972a041fab4cd5fb3541fc4ba)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:03.875206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:03.949386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca85cb037d0b44bfb5cc24f9a9bc463b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:03.950483 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca85cb037d0b44bfb5cc24f9a9bc463b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:03.964043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca85cb037d0b44bfb5cc24f9a9bc463b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:03.971697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca85cb037d0b44bfb5cc24f9a9bc463b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:03.972423 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d3d3fb00-6cd6-4f31-bfd3-d948f6fc8741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca85cb037d0b44bfb5cc24f9a9bc463b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.106613 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1349/2657] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:03 2022] DELETE /identity/v3/users/ca85cb037d0b44bfb5cc24f9a9bc463b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:04.112019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:04.112328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:04.112602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:04.133390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:04.134651 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:04.144819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f9014972a041fab4cd5fb3541fc4ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.145543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f9014972a041fab4cd5fb3541fc4ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.159181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f9014972a041fab4cd5fb3541fc4ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.159916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f9014972a041fab4cd5fb3541fc4ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.168952 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-6ce94082-1879-439a-b6dc-e69b045dc3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f9014972a041fab4cd5fb3541fc4ba'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:04.200213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1350/2658] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:04.216501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:04.218422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:04.268215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1309/2659] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:03 2022] DELETE /identity/v3/users/09f9014972a041fab4cd5fb3541fc4ba => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:04.272890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:04.273178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:04.273666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:04.291981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:04.293238 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:04.297067 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89d6b3-7e32-4c3c-a182-41163b68bbe3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:04.355937 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:04.356614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:04.361730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:04.361730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:04.362390 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1310/2660] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:04.370517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:04.372084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:04.373051 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:04.375391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:04.383310 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1351/2661] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:04.389502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6c5c4-8562-489a-a744-a9ced09e0617 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:04.395255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:04.396004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:04.396331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:04.396631 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:04.409781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:04.411258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:04.422721 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1311/2662] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:04.953713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:04.953713 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:04.970062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a594a4-5913-4b3c-b36e-f544236f5250 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:04.975159 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:04.976114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:04.976535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:04.976891 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] PATH_INFO: `/v3/projects/fe86a9041e0042b68458b93a6fde5006` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:04.978105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe86a9041e0042b68458b93a6fde5006)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:04.982752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:05.050922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:05.053195 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:05.081377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17326700-2681-4777-8c6f-18572f385ea3 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:05.086745 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:05.087856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:05.088312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:05.088770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] PATH_INFO: `/v3/projects/396703f2af0c43bf85afaf43309f2aa5` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:05.090358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=396703f2af0c43bf85afaf43309f2aa5)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:05.092420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:05.106355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe86a9041e0042b68458b93a6fde5006', 'request_id': 'req-7732aaf2-05c1-48ae-863a-4942e2642d5b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.207927 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe86a9041e0042b68458b93a6fde5006', 'request_id': 'req-7732aaf2-05c1-48ae-863a-4942e2642d5b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.214762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe86a9041e0042b68458b93a6fde5006', 'request_id': 'req-7732aaf2-05c1-48ae-863a-4942e2642d5b'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.227864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] The token cache is being invalidate because project fe86a9041e0042b68458b93a6fde5006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:05.228167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.228840 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.230460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396703f2af0c43bf85afaf43309f2aa5', 'request_id': 'req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.230617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1352/2663] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:04 2022] DELETE /identity/v3/projects/fe86a9041e0042b68458b93a6fde5006 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:05.231379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:05.233309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:05.297021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732aaf2-05c1-48ae-863a-4942e2642d5b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:05.333568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396703f2af0c43bf85afaf43309f2aa5', 'request_id': 'req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.339779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396703f2af0c43bf85afaf43309f2aa5', 'request_id': 'req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.352668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] The token cache is being invalidate because project 396703f2af0c43bf85afaf43309f2aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:05.352956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.353248 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:05.353589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.354133 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:05.354444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:05.354715 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] PATH_INFO: `/v3/users/d4ebfa9c169e4b7eb011aa93f1405781` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:05.355360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1312/2664] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:05 2022] DELETE /identity/v3/projects/396703f2af0c43bf85afaf43309f2aa5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:05.356402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:05.357912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:05.358871 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ebfa9c169e4b7eb011aa93f1405781)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:05.361065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:05.371359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2bc0-5e68-4d80-bd13-24d84f905f80 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:05.377370 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:05.378460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:05.378752 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:05.379030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] PATH_INFO: `/v3/users/592c345d2d484dfb95b310cd8f70e165` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:05.380298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] RBAC: Authorizing `identity:delete_user(user_id=592c345d2d484dfb95b310cd8f70e165)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:05.382239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:05.603764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ebfa9c169e4b7eb011aa93f1405781'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.604360 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ebfa9c169e4b7eb011aa93f1405781'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.616249 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ebfa9c169e4b7eb011aa93f1405781'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.622957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ebfa9c169e4b7eb011aa93f1405781'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.623601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ebfa9c169e4b7eb011aa93f1405781'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.683974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592c345d2d484dfb95b310cd8f70e165'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.684708 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592c345d2d484dfb95b310cd8f70e165'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.697524 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592c345d2d484dfb95b310cd8f70e165'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.698353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592c345d2d484dfb95b310cd8f70e165'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.706709 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f72968b7-8d40-44a3-9fc6-c534c91cabae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592c345d2d484dfb95b310cd8f70e165'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:05.711978 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1353/2665] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:05 2022] DELETE /identity/v3/users/d4ebfa9c169e4b7eb011aa93f1405781 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:05.713295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:05.714744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:05.778350 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf78f-4733-427d-8dc0-c454b8c49974 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:05.783774 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:05.784691 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:05.784989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:05.785247 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] PATH_INFO: `/v3/users/8a5b7be0974f4860bec18417fe230602` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.025221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5b7be0974f4860bec18417fe230602)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.025221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1313/2666] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:05 2022] DELETE /identity/v3/users/592c345d2d484dfb95b310cd8f70e165 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:06.036335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.037000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1314/2667] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:06.037000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.037000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.037000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deccab-76a3-4d0e-afa4-2cc8fe74e2a0 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.037336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.037336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.037336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.037336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.037672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.037672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.037672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1315/2668] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:06.037853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5b7be0974f4860bec18417fe230602'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.037853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5b7be0974f4860bec18417fe230602'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.053268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5b7be0974f4860bec18417fe230602'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.063090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5b7be0974f4860bec18417fe230602'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.063838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5b7be0974f4860bec18417fe230602'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.247919 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1354/2669] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:05 2022] DELETE /identity/v3/users/8a5b7be0974f4860bec18417fe230602 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:06.294644 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.296989 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.321335 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f52d1-3783-4b0f-8a3d-52be87bff28b service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.397460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.398355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.398640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.398918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] PATH_INFO: `/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.400214 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85bb63bb7c14d549b3daaf9ece2b673)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.402288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.534359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.535508 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.548301 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85bb63bb7c14d549b3daaf9ece2b673', 'request_id': 'req-50f2e7af-07c9-437c-a5a3-f115a110747d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.559088 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b317b4f-c2ac-4f09-9f9c-91b0553a69e4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.615882 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.616753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.617037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.617312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] PATH_INFO: `/v3/projects/df74131cba184b5095adf00962093262` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.618531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df74131cba184b5095adf00962093262)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.620479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.634963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85bb63bb7c14d549b3daaf9ece2b673', 'request_id': 'req-50f2e7af-07c9-437c-a5a3-f115a110747d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.641949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85bb63bb7c14d549b3daaf9ece2b673', 'request_id': 'req-50f2e7af-07c9-437c-a5a3-f115a110747d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.652815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] The token cache is being invalidate because project f85bb63bb7c14d549b3daaf9ece2b673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:06.653094 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.653746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50f2e7af-07c9-437c-a5a3-f115a110747d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.656124 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1316/2670] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:06 2022] DELETE /identity/v3/projects/f85bb63bb7c14d549b3daaf9ece2b673 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:06.661977 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bffea1-c2dd-40a8-a374-32c06f0595d7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.662319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bffea1-c2dd-40a8-a374-32c06f0595d7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.662594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bffea1-c2dd-40a8-a374-32c06f0595d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.681166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-29bffea1-c2dd-40a8-a374-32c06f0595d7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:06.682418 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-29bffea1-c2dd-40a8-a374-32c06f0595d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.691879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1317/2671] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:06.701297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.701581 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.701857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.721778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:06.723006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.731348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1318/2672] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:06.739206 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.746651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716c788-9857-44e3-9d39-f6c2c0db38c7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.754929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df74131cba184b5095adf00962093262', 'request_id': 'req-68432a5a-64c6-4359-a948-e0392932193f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.807510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.808118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.808400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.808674 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.809393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:06.809698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.811483 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.824014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1319/2673] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:54:06.824872 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.826666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.839241 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647366a-4098-4a7a-9157-8a71a254b4c0 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.844694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.845557 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.845840 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.846113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] PATH_INFO: `/v3/projects/72a7f2371e2149c2b744e9e7590550ba` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.847364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a7f2371e2149c2b744e9e7590550ba)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.848012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df74131cba184b5095adf00962093262', 'request_id': 'req-68432a5a-64c6-4359-a948-e0392932193f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.849417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.854111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df74131cba184b5095adf00962093262', 'request_id': 'req-68432a5a-64c6-4359-a948-e0392932193f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.866561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] The token cache is being invalidate because project df74131cba184b5095adf00962093262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:06.867438 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.868163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:06.869811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1355/2674] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:06 2022] DELETE /identity/v3/projects/df74131cba184b5095adf00962093262 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:06.870579 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:06.871839 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:06.885037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432a5a-64c6-4359-a948-e0392932193f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:06.890582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:06.891233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:06.891511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:06.897043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:06.897043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:06.897043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:06.956424 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a7f2371e2149c2b744e9e7590550ba', 'request_id': 'req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.009346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1356/2675] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:06 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:07.010191 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.011729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.030793 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a7f2371e2149c2b744e9e7590550ba', 'request_id': 'req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.036753 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a7f2371e2149c2b744e9e7590550ba', 'request_id': 'req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.050113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] The token cache is being invalidate because project 72a7f2371e2149c2b744e9e7590550ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:07.050437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.051130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.052727 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1320/2676] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:06 2022] DELETE /identity/v3/projects/72a7f2371e2149c2b744e9e7590550ba => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:07.053470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.056898 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.076711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b794dc-1077-4734-8790-d49b42bbb039 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.085850 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.086817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.087089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.087374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] PATH_INFO: `/v3/users/cdf57c0648c44129b345128969fa1ff5` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.088621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf57c0648c44129b345128969fa1ff5)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.090736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.125990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32e909-8d7a-480f-bbfd-0f8cc2bc3380 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.131031 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.131706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.131984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.132258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.132946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.134883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c85b5-8c0d-4dd5-a511-b97affce2bd7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.241650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1321/2677] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:07.243197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.243484 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.243763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.261978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:07.263231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.271744 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1322/2678] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:07.272638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.274512 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.286522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51e558-a9aa-4007-bfab-2fe9a01d0ffd None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.291347 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.292222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.292504 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.292789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] PATH_INFO: `/v3/users/a67605538b654ec4910eff76d297a992` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.293996 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67605538b654ec4910eff76d297a992)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.296566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.307346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf57c0648c44129b345128969fa1ff5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.308001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf57c0648c44129b345128969fa1ff5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.320103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf57c0648c44129b345128969fa1ff5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.326816 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf57c0648c44129b345128969fa1ff5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.327616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf57c0648c44129b345128969fa1ff5'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.431828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1357/2679] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:07 2022] DELETE /identity/v3/users/cdf57c0648c44129b345128969fa1ff5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:07.432808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.434140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.493293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67605538b654ec4910eff76d297a992'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.494025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67605538b654ec4910eff76d297a992'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.503773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd4d5-7d33-4039-aa27-e5c5c50c0bf2 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.508505 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.509224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.509499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.509868 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.510312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67605538b654ec4910eff76d297a992'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.511037 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67605538b654ec4910eff76d297a992'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.511281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.515103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2dcb1-a348-491d-b5a6-ebcdf3c15cd7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.520761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67605538b654ec4910eff76d297a992'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:07.528831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1358/2680] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:07.530886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.531321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.531602 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.551132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:07.552368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.561036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1359/2681] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:07.562014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.563374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.575498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd0b71-1d84-4c64-bfb4-e56c60f0faf5 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.580847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.581519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.581803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.582139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.582930 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.584524 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.595603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1360/2682] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:07.596308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.601944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1323/2683] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:07 2022] DELETE /identity/v3/users/a67605538b654ec4910eff76d297a992 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:07.603497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.607609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19de42b-5ab7-4ef5-b522-6c02906d059e admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.676878 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.677577 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.677826 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.678098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.678665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534887a7-6dc4-42da-bef2-f0eb3b59a5d7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.678863 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:07.679169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.680811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41568349-ec35-493f-aeb0-76c167ea9284 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.685671 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.686662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.686949 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.687227 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] PATH_INFO: `/v3/projects/9074ab69fecd4ab1815ffbf955b62405/users/38e8a8063a2c4f3688ca3c255be88056/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.696108 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1361/2684] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:07.697451 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.697720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.697997 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.712164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9074ab69fecd4ab1815ffbf955b62405, user_id=38e8a8063a2c4f3688ca3c255be88056, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.715173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.725189 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:07.726473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.741547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1362/2685] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:07.742647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.753797 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba644-980d-4b85-ab2c-72a4db02a8ca None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.759682 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.760341 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.760648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.760900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.761601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:07.763549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972d0b2-df41-40d0-8525-53db09b3954e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:07.802572 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1324/2686] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:07 2022] PUT /identity/v3/projects/9074ab69fecd4ab1815ffbf955b62405/users/38e8a8063a2c4f3688ca3c255be88056/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:07.803470 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:07.805185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.865153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1363/2687] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:07.865153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995555-d774-4bac-8651-294af8c6302f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.865153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995555-d774-4bac-8651-294af8c6302f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.865153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995555-d774-4bac-8651-294af8c6302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.875188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c502b7-e51f-4582-9a54-003a846c0364 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:07.896307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5f995555-d774-4bac-8651-294af8c6302f None None] MFA Rules not processed for user `38e8a8063a2c4f3688ca3c255be88056`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:07.897525 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5f995555-d774-4bac-8651-294af8c6302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:07.946544 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:07.947208 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:07.947497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:07.947772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:07.978192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1364/2688] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:07.979056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995555-d774-4bac-8651-294af8c6302f None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.014956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.016405 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.026421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1325/2689] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:08.027248 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.028736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.042550 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fb195-5ebd-4883-b36e-0409171b9928 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.046960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995555-d774-4bac-8651-294af8c6302f None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.049481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.050337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.050626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.050909 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] PATH_INFO: `/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.052102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c56d8a5b22e4c6cb74f218543c2d581)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.052892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.053696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.054018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.054174 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.054371 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.055156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.057751 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.181905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1365/2690] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:07 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:08.181905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.182600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.223729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c56d8a5b22e4c6cb74f218543c2d581', 'request_id': 'req-b39190c6-0f05-4621-a66f-eed655a7fe6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.260357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ee612-745f-4b3e-ad1e-b5795c556a35 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.312789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.313457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.313740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.314021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.328226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c56d8a5b22e4c6cb74f218543c2d581', 'request_id': 'req-b39190c6-0f05-4621-a66f-eed655a7fe6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.335764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c56d8a5b22e4c6cb74f218543c2d581', 'request_id': 'req-b39190c6-0f05-4621-a66f-eed655a7fe6a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.348163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] The token cache is being invalidate because project 7c56d8a5b22e4c6cb74f218543c2d581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:08.349013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.349681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.351614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1326/2691] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:08 2022] DELETE /identity/v3/projects/7c56d8a5b22e4c6cb74f218543c2d581 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:08.352316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.354822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.382251 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.383849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5bd09-99ef-43f2-88b0-6caec3132936 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.395103 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1366/2692] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:08.396419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cbde5-0a20-4e17-9a66-fcf6ef4f31c7 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.396700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cbde5-0a20-4e17-9a66-fcf6ef4f31c7 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.396977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cbde5-0a20-4e17-9a66-fcf6ef4f31c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.425568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-337cbde5-0a20-4e17-9a66-fcf6ef4f31c7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:08.426807 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-337cbde5-0a20-4e17-9a66-fcf6ef4f31c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.435494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1367/2693] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:08.435613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39190c6-0f05-4621-a66f-eed655a7fe6a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.441261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.442062 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.442449 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.442959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.443805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.444270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.444564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.444842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.446236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.464918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:08.465456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1327/2694] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:08.467446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.470260 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.477523 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1368/2695] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:08.482158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.492402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c24e3-b610-4caf-979c-661adb1878f3 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.501828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c50df-847a-4f01-9afb-c8ee08c0bf46 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.508281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.509218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.509497 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.509768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.510598 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:08.510848 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.512588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.518444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.519279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.519442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.519754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.520918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.522611 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.530117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1369/2696] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:08.530117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.530117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.537900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1328/2697] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:08.538409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.544214 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c17fc-0355-4b87-a64b-43c64e9e13fb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.550008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.550892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.551171 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.551444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] PATH_INFO: `/v3/projects/bda1f4ab03184482881e7a31ae1d7854` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.552425 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb3ec-ff90-4275-8fe9-010ac0954c58 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.552986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda1f4ab03184482881e7a31ae1d7854)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.555576 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.557860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.558586 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.558871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.559144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.559896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.562073 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.707463 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1329/2698] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:08 2022] POST /identity/v3/projects => generated 357 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:08.708255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.720008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda1f4ab03184482881e7a31ae1d7854', 'request_id': 'req-367651af-7fbd-4bda-99fc-36656e72c9b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.772832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47d2b7-cfe9-4fd2-900b-49c99706e1ab admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.777883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.778805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.779083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.779356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] PATH_INFO: `/v3/projects/ecf09dc89b40485c89f0df7292a91a35/users/83bb540224ec437a84de4e8a1ad6e87b/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.793610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf09dc89b40485c89f0df7292a91a35, user_id=83bb540224ec437a84de4e8a1ad6e87b, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.796385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080093d-f545-45ca-8de1-8588160bb385 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.847762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda1f4ab03184482881e7a31ae1d7854', 'request_id': 'req-367651af-7fbd-4bda-99fc-36656e72c9b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.853595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda1f4ab03184482881e7a31ae1d7854', 'request_id': 'req-367651af-7fbd-4bda-99fc-36656e72c9b4'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.865349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] The token cache is being invalidate because project bda1f4ab03184482881e7a31ae1d7854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:08.865634 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.866362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:08.867944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1370/2699] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:08 2022] DELETE /identity/v3/projects/bda1f4ab03184482881e7a31ae1d7854 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:08.868758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:08.870026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:08.932594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367651af-7fbd-4bda-99fc-36656e72c9b4 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:08.938307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:08.939014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.939303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.939581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:08.940312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:08.942260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e9dc4-214a-4c27-bca4-855b89c359ad admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:08.980791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1330/2700] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:08 2022] PUT /identity/v3/projects/ecf09dc89b40485c89f0df7292a91a35/users/83bb540224ec437a84de4e8a1ad6e87b/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:08.980791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41430-c0f3-4e94-8fac-affb6a6b1159 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:08.980791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41430-c0f3-4e94-8fac-affb6a6b1159 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:08.980791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41430-c0f3-4e94-8fac-affb6a6b1159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.000285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-92a41430-c0f3-4e94-8fac-affb6a6b1159 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:09.001567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92a41430-c0f3-4e94-8fac-affb6a6b1159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:09.073363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1331/2701] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:09.074685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.074978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.075258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.095055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1371/2702] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:08 2022] POST /identity/v3/users => generated 279 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.095233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.095427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.095706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.103350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] MFA Rules not processed for user `83bb540224ec437a84de4e8a1ad6e87b`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:09.104593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:09.114324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:09.115517 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:09.123873 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1372/2703] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:09.128595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.139052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23692c1-2cd6-4aca-b453-eb096cbd8dfd None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.143785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.144442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.144727 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.145008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.145838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.147518 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.158027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1373/2704] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:09.158898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.169268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75c5b1-0e9f-4c6b-8369-868b1cc17f04 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.174826 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1332/2705] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:09.175012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.175742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.175959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.176056 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.176304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.177033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:09.177345 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.179135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.187429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f3b6f-4d9c-4ca0-b4b0-6dcd1cfb96a5 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.192038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1374/2706] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:09.192908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.193285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.194016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.194396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.194728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.195015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:09.195612 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.197234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.210895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1333/2707] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:09.211055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03cf5-2faf-4d2a-b90f-260ac5277f10 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.211145 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.213942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.214650 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.214923 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.215205 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.215901 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.218022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.220672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c97c59-541f-4bfc-a679-6eaadda0baed admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.226129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.226942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.227225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.227509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.228199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.230107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.339493 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1375/2708] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.340326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.380311 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1334/2709] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.381917 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.422607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3c974-9889-4d1d-a168-871aa2671bc0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.447971 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f60de2-f79d-4025-84db-4b5dffb31265 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.453278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.453948 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.454252 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.454528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.455210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.457118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.480489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.481460 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.481689 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.482259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] PATH_INFO: `/v3/projects/99bbd9afd61a4fa3a887a8d5c1076a3c/users/53d09992a33145d1af08ae465dbf7a46/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.498167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bbd9afd61a4fa3a887a8d5c1076a3c, user_id=53d09992a33145d1af08ae465dbf7a46, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.501283 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.586794 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1335/2710] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.587440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1376/2711] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:09 2022] PUT /identity/v3/projects/99bbd9afd61a4fa3a887a8d5c1076a3c/users/53d09992a33145d1af08ae465dbf7a46/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:09.587960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.598348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.599021 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52856-cb27-4828-b1cc-214a8b573f6e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.657331 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.658142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.658460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.658776 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.659548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.660242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead86082-8dcf-4eed-afab-0243342cb879 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.661492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.664981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.665641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.669386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.669386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.669386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.669386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d22a-0c60-4662-8d3d-d101b8bcfcfc admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.679263 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1377/2712] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:09.680530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.680818 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.681094 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.708688 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] MFA Rules not processed for user `53d09992a33145d1af08ae465dbf7a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:09.710023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:09.760163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1336/2713] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/users => generated 282 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.760163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.769098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fb90-fe2c-4d68-bab9-34d93815ce83 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.774964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.775984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.776341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.776622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] PATH_INFO: `/v3/projects/5f1c4525102a46a58d4a5c5df733f955/users/5777e8fba0524a88b5c1b36713bf65c7/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.780200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1378/2714] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:09.781609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.792348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9159d6-e6c0-481e-be5d-b00c210005ef None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.794908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1c4525102a46a58d4a5c5df733f955, user_id=5777e8fba0524a88b5c1b36713bf65c7, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.797698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.797834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.798427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.798742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.799021 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.799757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.801358 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.811481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1379/2715] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:09.813167 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.822861 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228de21-f9e5-4688-8416-3cfd4d3152ed admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:09.828151 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:09.830829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:09.830829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:09.830829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:09.830829 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:09.831838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:09.894454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1337/2716] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:09 2022] PUT /identity/v3/projects/5f1c4525102a46a58d4a5c5df733f955/users/5777e8fba0524a88b5c1b36713bf65c7/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:09.895438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.958795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1380/2717] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:09 2022] POST /identity/v3/projects => generated 357 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:09.959621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:09.973676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fab39-6ca7-4f83-b984-578e9e855612 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.021163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46cc15-5e78-4a23-9d99-97b5c5e40003 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.025801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.026498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.026780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.027057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.027780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.029376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.031246 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.031915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.032198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.032476 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.033202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.034811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.041466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1381/2718] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:10.042355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.047869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1338/2719] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:10.047869 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.053041 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510d787-89fc-484a-bae7-2338ff2f8842 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.055088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41d32-0519-4d12-a3e2-135202f03ed5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.057676 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.058391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.058675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.058947 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.059737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.059893 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.060680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.060958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.061242 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] PATH_INFO: `/v3/projects/5f1c4525102a46a58d4a5c5df733f955/users/5777e8fba0524a88b5c1b36713bf65c7/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.062281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7990a5-da66-482f-b465-d4fc3d9d1b10 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.063274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1c4525102a46a58d4a5c5df733f955, user_id=5777e8fba0524a88b5c1b36713bf65c7, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.065907 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.076423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1339/2720] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:10 2022] PUT /identity/v3/projects/5f1c4525102a46a58d4a5c5df733f955/users/5777e8fba0524a88b5c1b36713bf65c7/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:10.077875 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.133640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1382/2721] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:10 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:10.134967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.135259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.135536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.141514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eef6ea-a647-4f4c-ba9e-52b2e3b11abc admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.146635 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.147563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.147852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.148134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] PATH_INFO: `/v3/projects/efd6fab5c21d4140a5bc407e035999bd/users/ad45aac8279d45209d3b4e3bc88fb729/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.163455 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd6fab5c21d4140a5bc407e035999bd, user_id=ad45aac8279d45209d3b4e3bc88fb729, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.164018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] MFA Rules not processed for user `5777e8fba0524a88b5c1b36713bf65c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:10.165209 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:10.166389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.234104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1383/2722] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:10.234993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.242510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1340/2723] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:10 2022] PUT /identity/v3/projects/efd6fab5c21d4140a5bc407e035999bd/users/ad45aac8279d45209d3b4e3bc88fb729/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:10.246575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.249191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:10.249368 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8104f-d466-40fa-a74e-47c878b461c0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.308022 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2459e2f-971c-4720-8be6-a41a5a57b82e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.308640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.309292 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.310242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.310528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.311796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.313377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.314036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.314144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb381f1d-c106-49c4-ae90-5569410da497 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.314486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.314779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.328703 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1384/2724] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:10.330359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.330596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.330805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.358109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] MFA Rules not processed for user `ad45aac8279d45209d3b4e3bc88fb729`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:10.360485 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:10.381616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.383391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.394013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1341/2725] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:10.394775 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.405445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02dc0c-9b4d-4201-bbc0-f83e1a5e2b7d service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.410643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.411585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.411863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.412279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] PATH_INFO: `/v3/projects/ed877f67d66349bbb2244c4af4a2e214/users/a8a32f1293714517b3c95b02d993d211/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.428156 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed877f67d66349bbb2244c4af4a2e214, user_id=a8a32f1293714517b3c95b02d993d211, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.431018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.431273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1385/2726] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:10.445892 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.447408 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:10.461179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf7ad0-491c-4d87-9739-2a7840a0c46b None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.465013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.465657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.465938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.466228 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.476284 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.477770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.488649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1386/2727] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:10.557599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1342/2728] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:10 2022] PUT /identity/v3/projects/ed877f67d66349bbb2244c4af4a2e214/users/a8a32f1293714517b3c95b02d993d211/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:10.561034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.571782 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0821fd-c97f-4055-84bc-6068591fa28c service cinder] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.636680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.637355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.637639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.637916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.638696 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.641156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4d6e9-b7fd-4498-b148-349c6d3dfb11 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.655771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1387/2729] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:10.657454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:10.668378 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f61c-5464-4d83-9a74-19dd9a20e300 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:10.673625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:10.674608 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.675038 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.675379 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] PATH_INFO: `/v3/projects/ed877f67d66349bbb2244c4af4a2e214/users/a8a32f1293714517b3c95b02d993d211/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.677466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed877f67d66349bbb2244c4af4a2e214, user_id=a8a32f1293714517b3c95b02d993d211, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:10.680235 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:10.690975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1343/2730] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:10 2022] PUT /identity/v3/projects/ed877f67d66349bbb2244c4af4a2e214/users/a8a32f1293714517b3c95b02d993d211/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:10.700918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:10.701593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:10.702489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:10.732317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] MFA Rules not processed for user `a8a32f1293714517b3c95b02d993d211`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:10.733778 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:10.803339 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1388/2731] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:11.358902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:11.413230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18791fac-bb51-4229-9e30-71cc6d28a59f admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:11.417052 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:11.417734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:11.418018 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:11.418333 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:11.429693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:11.430755 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:11.441034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1344/2732] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:11 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:11.830916 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:11.831627 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:11.840135 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48fe6-3620-41f9-ba5e-37fc95fd9960 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:11.844717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:11.845371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:11.845652 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:11.845927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:11.894946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad9e9-e49b-48d8-8f4b-758a31d56d13 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:11.900144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5777e8fba0524a88b5c1b36713bf65c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f1c4525102a46a58d4a5c5df733f955', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:11.900812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:11.901102 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:11.901379 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] PATH_INFO: `/v3/domains` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:11.902111 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:11.902459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:11.904142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:11.906602 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:11.908057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:11.916714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1389/2733] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:54:11.917234 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1345/2734] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:11 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:11.921639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:11.933252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294f7d6-0cd3-445f-9463-a8e51e3cc62b tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:11.938777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5777e8fba0524a88b5c1b36713bf65c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f1c4525102a46a58d4a5c5df733f955', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:11.939444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:11.941587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:11.941587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:11.941587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:11.942609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:12.090828 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1390/2735] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:11 2022] POST /identity/v3/projects => generated 354 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:12.095584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:12.106459 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fadf2c-31c7-4c52-9e04-244f668ab7ac service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:12.164083 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5777e8fba0524a88b5c1b36713bf65c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f1c4525102a46a58d4a5c5df733f955', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:12.164783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:12.165190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:12.165490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:12.166249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:12.171646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:12.171646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:12.180491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1346/2736] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:12.392905 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:12.404281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fa228-da35-4cf1-9026-2aebfad3ec27 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:12.413907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5777e8fba0524a88b5c1b36713bf65c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f1c4525102a46a58d4a5c5df733f955', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:12.414847 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:12.415136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:12.417945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] PATH_INFO: `/v3/projects/5bd011e4f9a3486ab72151a8c6522640` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:12.417945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5bd011e4f9a3486ab72151a8c6522640)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:12.418765 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:12.539652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd011e4f9a3486ab72151a8c6522640', 'request_id': 'req-d07eedc6-fd6c-45aa-ab26-af98827d8281'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:12.686572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd011e4f9a3486ab72151a8c6522640', 'request_id': 'req-d07eedc6-fd6c-45aa-ab26-af98827d8281'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:12.692812 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd011e4f9a3486ab72151a8c6522640', 'request_id': 'req-d07eedc6-fd6c-45aa-ab26-af98827d8281'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:12.704981 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] The token cache is being invalidate because project 5bd011e4f9a3486ab72151a8c6522640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:12.705282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:12.705946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d07eedc6-fd6c-45aa-ab26-af98827d8281 tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:12.707539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1391/2737] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:12 2022] DELETE /identity/v3/projects/5bd011e4f9a3486ab72151a8c6522640 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:13.460827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:13.462617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:13.544119 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b229-4a73-4313-80ad-ac7518a0cbeb tempest-VolumeQuotaClassesTest-1486715477 tempest-VolumeQuotaClassesTest-1486715477-project-admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:13.608519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:13.609408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:13.609696 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:13.609974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] PATH_INFO: `/v3/users/83bb540224ec437a84de4e8a1ad6e87b` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:13.611289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bb540224ec437a84de4e8a1ad6e87b)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:13.613261 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:13.860621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bb540224ec437a84de4e8a1ad6e87b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:13.861281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bb540224ec437a84de4e8a1ad6e87b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:13.872854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bb540224ec437a84de4e8a1ad6e87b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:13.874107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bb540224ec437a84de4e8a1ad6e87b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:13.893912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bb540224ec437a84de4e8a1ad6e87b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:13.996230 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1347/2738] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:13 2022] DELETE /identity/v3/users/83bb540224ec437a84de4e8a1ad6e87b => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:14.001359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:14.001820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:14.001925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:14.019970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:14.021184 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:14.079328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1392/2739] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:14.089774 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:14.091190 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:14.148144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b9b4-0ccf-4a20-b5df-72448f5e6cfa admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:14.151731 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:14.152366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:14.152642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:14.152913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:14.162249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:14.163670 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:14.173389 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1348/2740] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:14.619635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:14.621104 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:14.635105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76a262-526c-4312-856f-ca860e1e6e9c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:14.641070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:14.641865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:14.642145 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:14.642621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] PATH_INFO: `/v3/projects/ecf09dc89b40485c89f0df7292a91a35` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:14.643822 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf09dc89b40485c89f0df7292a91a35)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:14.645885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:14.802907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf09dc89b40485c89f0df7292a91a35', 'request_id': 'req-e6088e33-29c5-4206-bd5a-2be10c44a2a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:14.898766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf09dc89b40485c89f0df7292a91a35', 'request_id': 'req-e6088e33-29c5-4206-bd5a-2be10c44a2a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:14.905390 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf09dc89b40485c89f0df7292a91a35', 'request_id': 'req-e6088e33-29c5-4206-bd5a-2be10c44a2a6'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:14.916153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] The token cache is being invalidate because project ecf09dc89b40485c89f0df7292a91a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:14.916442 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:14.917065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:14.918681 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1393/2741] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:14 2022] DELETE /identity/v3/projects/ecf09dc89b40485c89f0df7292a91a35 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:14.923460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:14.933297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7d8e-6a77-4747-8219-b60ddf4a36b9 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:14.997028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:14.997891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:14.998171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:14.998617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] PATH_INFO: `/v3/users/5777e8fba0524a88b5c1b36713bf65c7` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:15.000171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5777e8fba0524a88b5c1b36713bf65c7)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:15.002087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:15.253787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5777e8fba0524a88b5c1b36713bf65c7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:15.254422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5777e8fba0524a88b5c1b36713bf65c7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:15.266870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5777e8fba0524a88b5c1b36713bf65c7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:15.267514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5777e8fba0524a88b5c1b36713bf65c7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:15.274188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-fd4ecc70-29cb-4607-b26c-e8cc7b52ddea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5777e8fba0524a88b5c1b36713bf65c7'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:15.374000 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1349/2742] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:14 2022] DELETE /identity/v3/users/5777e8fba0524a88b5c1b36713bf65c7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:15.855421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:15.857142 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:15.872374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088e33-29c5-4206-bd5a-2be10c44a2a6 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:15.929349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:15.930394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:15.930888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:15.930888 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] PATH_INFO: `/v3/projects/5f1c4525102a46a58d4a5c5df733f955` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:15.932398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1c4525102a46a58d4a5c5df733f955)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:15.934323 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.041968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1c4525102a46a58d4a5c5df733f955', 'request_id': 'req-6107351a-22eb-4e68-926c-590c93296e3e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:16.122050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1c4525102a46a58d4a5c5df733f955', 'request_id': 'req-6107351a-22eb-4e68-926c-590c93296e3e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:16.128924 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1c4525102a46a58d4a5c5df733f955', 'request_id': 'req-6107351a-22eb-4e68-926c-590c93296e3e'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:16.140749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] The token cache is being invalidate because project 5f1c4525102a46a58d4a5c5df733f955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:16.141036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:16.141777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6107351a-22eb-4e68-926c-590c93296e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:16.143440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1394/2743] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:15 2022] DELETE /identity/v3/projects/5f1c4525102a46a58d4a5c5df733f955 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:16.148519 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.148982 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.149087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.167251 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:16.168540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:16.227819 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1350/2744] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:16.236906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.237193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.237474 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.255740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:16.256946 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:16.264992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1395/2745] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:16.269365 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.279148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb00f-7ee7-41da-9841-a80d1f635cd6 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.284629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.285315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.285593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.285863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.286634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:16.286930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.288530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.300824 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1351/2746] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:16.305386 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.315179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df2fd-9e21-49bb-b741-e36825fd3e0a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.320233 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.320886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.321170 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.321446 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.322146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.324073 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.403708 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1396/2747] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:16.409033 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.419363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ef61-eafd-4929-8bd5-d22ab44ade93 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.480740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.481419 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.481958 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.482262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.483063 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.484970 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.609941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1352/2748] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:16.614543 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.626131 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239805-e945-45eb-afcf-51d2b1de25a3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.630777 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.631430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.631716 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.632019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.632898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.634527 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.645616 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1397/2749] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:16.649975 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.660371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec3c5-2926-4250-9adc-5fc29c538ca1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.664988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.665668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.665950 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.666312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.667045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.668742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a95dd7-1c4d-4be6-8622-7f65a0b974f9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.681613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1353/2750] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:16.686666 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:16.697422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936765-53bc-4e36-aca7-172cee6f9311 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:16.702395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:16.703294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.703578 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.703851 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] PATH_INFO: `/v3/projects/ee70b5646f6b4115934263c9af4d4212/users/6314b570c1dd4908aa22205b50f98865/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.725034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee70b5646f6b4115934263c9af4d4212, user_id=6314b570c1dd4908aa22205b50f98865, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:16.725034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:16.834875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1398/2751] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:16 2022] PUT /identity/v3/projects/ee70b5646f6b4115934263c9af4d4212/users/6314b570c1dd4908aa22205b50f98865/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:16.846380 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:16.846666 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:16.846939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:16.875226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] MFA Rules not processed for user `6314b570c1dd4908aa22205b50f98865`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:16.876438 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:16.939805 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1354/2752] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:16.944091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.005264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c3d7-6ba7-421f-bac4-e2feaeffc286 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.010001 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.010694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.010987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.011265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.011974 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.013902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.105400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1399/2753] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:17.110734 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.121792 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349bb33-d2ef-48d0-ab27-2b5dbf2c63d3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.185059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.185757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.186046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.186361 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.187559 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.189249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.294237 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1355/2754] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:17 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:17.298320 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.308630 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0447fc-0fef-4a3f-81db-d8a5cf01a9d2 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.313246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.313907 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.319346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.319346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.319346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.319346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.327842 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1400/2755] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:54:17.332131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.343196 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0756340-3588-483a-8372-671cad1e8ea8 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.348131 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.350533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.350533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.350533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] PATH_INFO: `/v3/projects/4fbd4f54c52c45ba90913b732e874854/users/efb6fd0851a24dad86a8472808203d69/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.365430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbd4f54c52c45ba90913b732e874854, user_id=efb6fd0851a24dad86a8472808203d69, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.368292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.579779 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1356/2756] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:17 2022] PUT /identity/v3/projects/4fbd4f54c52c45ba90913b732e874854/users/efb6fd0851a24dad86a8472808203d69/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:17.585647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.596764 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c002184-2b46-4f49-9b82-8bb656b14eb0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.658638 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.659313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.659603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.659884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.660614 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.662282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d5285-d527-4eb1-a5b7-3dff29282fad admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.672404 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1401/2757] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:17.676915 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.687801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d09f41-e10e-49d3-ade6-0a6e95fbeebe admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.692358 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.693226 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.693509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.693811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] PATH_INFO: `/v3/projects/4fbd4f54c52c45ba90913b732e874854/users/efb6fd0851a24dad86a8472808203d69/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.696617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbd4f54c52c45ba90913b732e874854, user_id=efb6fd0851a24dad86a8472808203d69, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.699415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.711036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1357/2758] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:17 2022] PUT /identity/v3/projects/4fbd4f54c52c45ba90913b732e874854/users/efb6fd0851a24dad86a8472808203d69/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:17.721507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.721805 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.722087 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.749376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] MFA Rules not processed for user `efb6fd0851a24dad86a8472808203d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:17.750881 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:17.829986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1402/2759] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:17.841354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:17.843726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:17.909044 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4695-d557-44b3-add2-41dfc693efe1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:17.912920 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:17.913578 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:17.913864 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:17.914144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:17.924822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:17.926341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:17.936707 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1358/2760] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:18.121312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:18.133206 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f792e-0bef-444f-a03d-0b37c8dd1aea None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:18.137140 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:18.137942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:18.138361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:18.138744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:18.199132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:18.200749 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305204c-2473-49e2-a85c-11c7ca8e3e79 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:18.210280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1403/2761] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:18.817946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:18.894883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9a89-15ba-4ba5-8d9b-4aaed322bcba service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:18.900171 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:18.901136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:18.901591 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:18.901979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] PATH_INFO: `/v3/users/6314b570c1dd4908aa22205b50f98865` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:18.903267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6314b570c1dd4908aa22205b50f98865)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:18.906292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:19.149340 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6314b570c1dd4908aa22205b50f98865'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:19.150344 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6314b570c1dd4908aa22205b50f98865'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:19.163350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6314b570c1dd4908aa22205b50f98865'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:19.164100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6314b570c1dd4908aa22205b50f98865'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:19.173659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6314b570c1dd4908aa22205b50f98865'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:19.248815 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1359/2762] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:18 2022] DELETE /identity/v3/users/6314b570c1dd4908aa22205b50f98865 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:19.256699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:19.257529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:19.258122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:19.277542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:19.279784 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:19.364657 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1404/2763] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:19.383297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:19.385117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:19.460113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eaefb-8f95-4362-9f52-c269e01aa35d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:19.469413 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:19.470548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:19.471040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:19.471534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:19.487827 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:19.489540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:19.500565 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1360/2764] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:20.048564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:20.050331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:20.075581 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a42f-3013-41bd-9bc5-1bf25477c313 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:20.085499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:20.087037 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:20.088998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:20.088998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] PATH_INFO: `/v3/projects/ee70b5646f6b4115934263c9af4d4212` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:20.090340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee70b5646f6b4115934263c9af4d4212)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:20.094086 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:20.266290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee70b5646f6b4115934263c9af4d4212', 'request_id': 'req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.344537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee70b5646f6b4115934263c9af4d4212', 'request_id': 'req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.351016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee70b5646f6b4115934263c9af4d4212', 'request_id': 'req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.363635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] The token cache is being invalidate because project ee70b5646f6b4115934263c9af4d4212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:20.363929 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.364652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.367546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1405/2765] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:20 2022] DELETE /identity/v3/projects/ee70b5646f6b4115934263c9af4d4212 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:20.372692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:20.383447 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a805e5a-448b-4717-8ca2-a9796fa68ae5 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:20.457221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:20.458319 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:20.458606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:20.458885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] PATH_INFO: `/v3/users/efb6fd0851a24dad86a8472808203d69` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:20.460084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb6fd0851a24dad86a8472808203d69)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:20.461993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:20.764423 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb6fd0851a24dad86a8472808203d69'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.765210 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb6fd0851a24dad86a8472808203d69'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.777097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb6fd0851a24dad86a8472808203d69'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.777778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb6fd0851a24dad86a8472808203d69'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.785099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-f97ee62d-6d25-4b41-83a3-58b9a12a07ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb6fd0851a24dad86a8472808203d69'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:20.887454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1361/2766] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:20 2022] DELETE /identity/v3/users/efb6fd0851a24dad86a8472808203d69 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:21.323512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:21.325084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:21.352049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe9dc7-9ab0-41c3-afd1-22658cb11e20 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:21.411009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:21.411875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:21.412157 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:21.412437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] PATH_INFO: `/v3/projects/4fbd4f54c52c45ba90913b732e874854` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:21.413628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fbd4f54c52c45ba90913b732e874854)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:21.415620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:21.645542 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fbd4f54c52c45ba90913b732e874854', 'request_id': 'req-910d82eb-d72d-43c9-ba94-193bce4b4842'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:21.767865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fbd4f54c52c45ba90913b732e874854', 'request_id': 'req-910d82eb-d72d-43c9-ba94-193bce4b4842'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:21.773719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fbd4f54c52c45ba90913b732e874854', 'request_id': 'req-910d82eb-d72d-43c9-ba94-193bce4b4842'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:21.785498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] The token cache is being invalidate because project 4fbd4f54c52c45ba90913b732e874854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:21.785793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:21.786567 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-910d82eb-d72d-43c9-ba94-193bce4b4842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:21.788169 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1406/2767] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:21 2022] DELETE /identity/v3/projects/4fbd4f54c52c45ba90913b732e874854 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:21.793078 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:21.793534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:21.793616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:21.813865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:21.815100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:21.882693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1362/2768] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:21.887440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:21.887741 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:21.888019 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:21.907397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:21.908735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:21.916999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1407/2769] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:21.924267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:21.934749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac89c8-9529-4774-892e-810eed910688 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:21.941255 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:21.942010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:21.942324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:21.942606 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:21.943393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:21.943880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:21.949718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:21.962126 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1363/2770] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:21.963328 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:21.973445 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38855e4d-4995-425b-9cbe-257daf224098 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:21.979624 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:21.980307 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:21.980593 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:21.980877 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:21.981845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:21.988092 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.112837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1408/2771] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:21 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:22.117348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.128981 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac5841-ac9c-4c89-ae1e-14dbfb4df928 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.191200 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.191877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.192163 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.192446 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.193139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.195084 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.300199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1364/2772] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:22.304384 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.315009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4087d-ebcd-4e36-9dc7-d16dd368ec76 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.319771 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.320664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.320950 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.321245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.321960 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.323752 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.336331 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1409/2773] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:22.340654 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.351681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71132a3-7c1b-446f-95bf-4bc4784598f0 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.356320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.357240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.358173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.358453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.359217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.360807 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e45c3-36bf-4cbd-a15c-48601b04078d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.376306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1365/2774] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:22.387049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.397833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a67c4a-5176-4ecf-b4e6-986e79f54910 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.404720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.405837 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.406117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.406431 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] PATH_INFO: `/v3/projects/39dadc26268647eebd515abf7c80f5bf/users/9cd124ce49c94038bf476df4cca3905a/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.425860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dadc26268647eebd515abf7c80f5bf, user_id=9cd124ce49c94038bf476df4cca3905a, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.428590 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.518164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1410/2775] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:22 2022] PUT /identity/v3/projects/39dadc26268647eebd515abf7c80f5bf/users/9cd124ce49c94038bf476df4cca3905a/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:22.529688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.530076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.531005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.558533 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] MFA Rules not processed for user `9cd124ce49c94038bf476df4cca3905a`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:22.559741 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.614473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.616137 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:22.625942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1366/2776] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:22.630299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.681259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656f6e-a03c-4583-ad68-a261569cac62 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.687537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.688410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.688694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.689028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] PATH_INFO: `/v3/users/38e8a8063a2c4f3688ca3c255be88056` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.690304 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e8a8063a2c4f3688ca3c255be88056)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.692262 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.693350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30baf2-768f-4b37-956e-003d093bf04f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.698074 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.698757 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.703081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.703081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.703081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.703081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:22.866409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1367/2777] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:22.870268 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:22.905225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e8a8063a2c4f3688ca3c255be88056'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:22.905934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e8a8063a2c4f3688ca3c255be88056'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:22.918213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e8a8063a2c4f3688ca3c255be88056'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:22.925748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e8a8063a2c4f3688ca3c255be88056'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:22.926486 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9e4cf519-c480-4376-88da-9cbb47beda61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e8a8063a2c4f3688ca3c255be88056'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:22.935604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3443f8-75aa-4504-aacc-03feacb2fa73 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:22.940399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:22.941072 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:22.941359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:22.941640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:22.943450 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:22.944295 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.007903 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1411/2778] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:22 2022] DELETE /identity/v3/users/38e8a8063a2c4f3688ca3c255be88056 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:23.012349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.012641 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.012918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.031701 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:23.032958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.037312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1368/2779] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:22 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:23.041383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.043151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.055933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89c545-c51e-402a-b6dd-17c53c0202fd admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.102280 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1412/2780] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:54:23.112857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.114418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.123218 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.123980 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.124416 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.124700 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.125445 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.127103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.137437 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1369/2781] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:23.141900 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.153773 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf005-9d4b-4d13-9b3a-fa0e5906a2f3 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.160030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.161002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.161221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.161498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] PATH_INFO: `/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f/users/6e0ef76ce3ce4544aae687a98851309e/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.175973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448546a-2584-4736-a95b-b8f73aaadd82 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.177804 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7014ed91d7ac41908bbdaa53629bfd2f, user_id=6e0ef76ce3ce4544aae687a98851309e, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.180096 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.180502 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.180750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.181034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.181309 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.192362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.193832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.204403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1413/2782] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:54:23.272159 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1370/2783] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:23 2022] PUT /identity/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f/users/6e0ef76ce3ce4544aae687a98851309e/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:23.276243 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.286760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834618de-d729-4f9d-bb4a-188779552ed3 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.343457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.344122 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.344391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.344663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.345447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.347124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51919e9b-7bb9-4b0d-9ce1-7855e140a94d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.357207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1414/2784] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:54:23.361817 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.372271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389025-715a-4679-9c7d-93a6e6d434f7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.377500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.378596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.378878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.379155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] PATH_INFO: `/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f/users/6e0ef76ce3ce4544aae687a98851309e/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.381109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7014ed91d7ac41908bbdaa53629bfd2f, user_id=6e0ef76ce3ce4544aae687a98851309e, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.383925 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.396284 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1371/2785] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:23 2022] PUT /identity/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f/users/6e0ef76ce3ce4544aae687a98851309e/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:23.407898 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.408190 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.408467 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.437883 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] MFA Rules not processed for user `6e0ef76ce3ce4544aae687a98851309e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:23.439136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.517146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1415/2786] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:23.528870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.531961 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.593845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614833b4-f7bb-414e-83f3-e26a21fddf85 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.597613 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.598285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.598568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.598843 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.600597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.655632 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:23.657200 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:23.664716 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.666735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca3e6-46e5-40cf-b6dc-4142be26f04b service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.677257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1372/2787] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:23.728063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddf320-6240-4cad-b970-1e35ce67a4b1 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:23.733522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:23.734418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.734756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.735040 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] PATH_INFO: `/v3/projects/9074ab69fecd4ab1815ffbf955b62405` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:23.736245 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9074ab69fecd4ab1815ffbf955b62405)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:23.738294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:23.869213 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9074ab69fecd4ab1815ffbf955b62405', 'request_id': 'req-1e028dda-5095-4fb8-b649-f2df97d5d818'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:23.961640 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9074ab69fecd4ab1815ffbf955b62405', 'request_id': 'req-1e028dda-5095-4fb8-b649-f2df97d5d818'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:23.968005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9074ab69fecd4ab1815ffbf955b62405', 'request_id': 'req-1e028dda-5095-4fb8-b649-f2df97d5d818'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:23.979754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] The token cache is being invalidate because project 9074ab69fecd4ab1815ffbf955b62405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:23.988420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:23.988420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-1e028dda-5095-4fb8-b649-f2df97d5d818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:23.988420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1416/2788] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:23 2022] DELETE /identity/v3/projects/9074ab69fecd4ab1815ffbf955b62405 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:23.988672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:23.988672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:23.988672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.006271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:24.007475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:24.068231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1373/2789] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:24.077735 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.078077 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.078415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.104221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:24.105855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:24.115549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1417/2790] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:24.120036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.130384 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52887f-8978-4dc1-8171-4b314c76cad9 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.135216 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.135878 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.136153 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.136427 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.137152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:24.137451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.139388 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.150988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1374/2791] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:24.155541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.164596 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a742e-6a63-4dc6-90e8-e7dbc598f0ee None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.169282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.170587 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.170973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.171105 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.171795 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.173685 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.268993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1418/2792] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:24 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:24.275374 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.283667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997abdc2-e4f3-4a12-92fd-849fd8861552 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.366870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.367521 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.367787 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.368069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.368756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.370839 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.505382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1375/2793] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:24 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:24.510300 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.521055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb0da-43b1-44db-997b-a2080e61e297 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.525845 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.526544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.526831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.527110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.527841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.530158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.545829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.551893 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1419/2794] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:24.556487 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eedd3f-1586-407d-a61d-6e3238612883 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.561991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.562681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.562965 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.563240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.563967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.565626 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119abed-a4aa-4d21-9c8c-6a8caba65572 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.576770 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1376/2795] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:54:24.581132 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:24.598513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e44b-15bd-4634-a5e0-d92754b532f5 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:24.604480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:24.605329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.605621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.605989 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] PATH_INFO: `/v3/projects/50cf973c340644c5aee5349945ec9434/users/e620365b2a2948e291015e81b41bd3c1/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.625803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cf973c340644c5aee5349945ec9434, user_id=e620365b2a2948e291015e81b41bd3c1, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:24.628627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:24.762034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1420/2796] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:24 2022] PUT /identity/v3/projects/50cf973c340644c5aee5349945ec9434/users/e620365b2a2948e291015e81b41bd3c1/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:24.772895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:24.773185 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:24.773490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:24.801456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] MFA Rules not processed for user `e620365b2a2948e291015e81b41bd3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:24.802685 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:24.878259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1377/2797] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:25.187935 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:25.190874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:25.264568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c5c07-b3fc-48f1-945b-d423182fdb72 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:25.268236 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:25.268973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:25.269260 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:25.269534 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:25.278680 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:25.280264 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7dfdb-19df-4127-b13c-2f1d5d57f18b service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:25.289750 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1421/2798] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:33.235152 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:33.236563 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:33.307420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6551ada-648a-41ef-9795-423c928d9883 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:33.313475 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:33.314473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:33.314737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:33.315014 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] PATH_INFO: `/v3/users/9cd124ce49c94038bf476df4cca3905a` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:33.316324 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd124ce49c94038bf476df4cca3905a)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:33.318362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:33.609594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd124ce49c94038bf476df4cca3905a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:33.610298 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd124ce49c94038bf476df4cca3905a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:33.623371 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd124ce49c94038bf476df4cca3905a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:33.624106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd124ce49c94038bf476df4cca3905a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:33.637337 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd124ce49c94038bf476df4cca3905a'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:33.767556 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1378/2799] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:33 2022] DELETE /identity/v3/users/9cd124ce49c94038bf476df4cca3905a => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:33.772274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:33.772757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:33.772841 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:33.790424 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:33.791621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:33.849054 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1422/2800] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:33.861853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:33.863610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:33.920789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03140-ce2e-465a-a780-f7623a3d480e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:33.925061 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:33.925758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:33.926040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:33.926395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:33.938851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:33.940315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:33.954109 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1379/2801] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:34.438045 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:34.439500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:34.453865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf82b-6dfa-4be9-b72c-b25bd2f93cfc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:34.460528 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:34.462312 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:34.462603 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:34.463266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] PATH_INFO: `/v3/projects/39dadc26268647eebd515abf7c80f5bf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:34.464502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=39dadc26268647eebd515abf7c80f5bf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:34.467135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:34.634673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39dadc26268647eebd515abf7c80f5bf', 'request_id': 'req-f1c013d7-47cd-4835-b166-540875b2cffe'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:34.720999 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39dadc26268647eebd515abf7c80f5bf', 'request_id': 'req-f1c013d7-47cd-4835-b166-540875b2cffe'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:34.729070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39dadc26268647eebd515abf7c80f5bf', 'request_id': 'req-f1c013d7-47cd-4835-b166-540875b2cffe'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:34.741720 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] The token cache is being invalidate because project 39dadc26268647eebd515abf7c80f5bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:34.742055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:34.742599 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:34.744164 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1423/2802] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:34 2022] DELETE /identity/v3/projects/39dadc26268647eebd515abf7c80f5bf => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:34.749402 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:34.760273 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3c5c-d4f1-4052-ae0b-c673c75ded0d service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:34.819111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:34.819993 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:34.820269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:34.820540 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] PATH_INFO: `/v3/users/6e0ef76ce3ce4544aae687a98851309e` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:34.821738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0ef76ce3ce4544aae687a98851309e)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:34.823788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:35.084942 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0ef76ce3ce4544aae687a98851309e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.085680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0ef76ce3ce4544aae687a98851309e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.099740 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0ef76ce3ce4544aae687a98851309e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.100467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0ef76ce3ce4544aae687a98851309e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.108308 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0ef76ce3ce4544aae687a98851309e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.191551 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1380/2803] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:34 2022] DELETE /identity/v3/users/6e0ef76ce3ce4544aae687a98851309e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:35.632182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:35.633748 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:35.647290 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c013d7-47cd-4835-b166-540875b2cffe admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:35.713035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:35.713904 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:35.714188 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:35.714498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] PATH_INFO: `/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:35.716601 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7014ed91d7ac41908bbdaa53629bfd2f)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:35.718769 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:35.814934 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7014ed91d7ac41908bbdaa53629bfd2f', 'request_id': 'req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.926673 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7014ed91d7ac41908bbdaa53629bfd2f', 'request_id': 'req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.933207 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7014ed91d7ac41908bbdaa53629bfd2f', 'request_id': 'req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.944725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] The token cache is being invalidate because project 7014ed91d7ac41908bbdaa53629bfd2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:35.945016 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.945674 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-7445790b-02ed-4d8f-a16d-e4afc9c4fc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:35.947355 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1424/2804] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:35 2022] DELETE /identity/v3/projects/7014ed91d7ac41908bbdaa53629bfd2f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:47.915992 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:47.917507 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:47.932117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c92381-774d-48ac-bc71-38c19d2debf7 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:47.991306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:47.992114 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:47.992391 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:47.992662 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] PATH_INFO: `/v3/users/53d09992a33145d1af08ae465dbf7a46` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:47.994003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d09992a33145d1af08ae465dbf7a46)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:48.010334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:48.210134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d09992a33145d1af08ae465dbf7a46'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:48.210932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d09992a33145d1af08ae465dbf7a46'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:48.223539 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d09992a33145d1af08ae465dbf7a46'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:48.224188 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d09992a33145d1af08ae465dbf7a46'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:48.231668 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d09992a33145d1af08ae465dbf7a46'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:48.326910 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1381/2805] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:47 2022] DELETE /identity/v3/users/53d09992a33145d1af08ae465dbf7a46 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:48.335821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:48.335821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:48.335821 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:48.350550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:48.351740 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:48.412270 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1425/2806] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:48.422281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:48.423673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:48.490259 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6a1eb-fd4a-4719-9364-7c583b5738f6 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:48.495326 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:48.496041 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:48.496856 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:48.497279 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:48.508640 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:48.510404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:48.521593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1382/2807] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:48.968838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:48.970306 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:48.983032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c699723-6394-48aa-afe6-d6486b6a0a5a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:48.987643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:48.988498 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:48.988781 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:48.989060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] PATH_INFO: `/v3/projects/99bbd9afd61a4fa3a887a8d5c1076a3c` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:48.990278 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bbd9afd61a4fa3a887a8d5c1076a3c)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:48.992416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:49.102776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bbd9afd61a4fa3a887a8d5c1076a3c', 'request_id': 'req-6f417133-cc88-464c-8c13-97ae59e8402f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.176655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bbd9afd61a4fa3a887a8d5c1076a3c', 'request_id': 'req-6f417133-cc88-464c-8c13-97ae59e8402f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.182519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bbd9afd61a4fa3a887a8d5c1076a3c', 'request_id': 'req-6f417133-cc88-464c-8c13-97ae59e8402f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.194516 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] The token cache is being invalidate because project 99bbd9afd61a4fa3a887a8d5c1076a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:49.194803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.195489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.197066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1426/2808] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:48 2022] DELETE /identity/v3/projects/99bbd9afd61a4fa3a887a8d5c1076a3c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:49.201694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:49.212373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ecfec-8fd8-47ad-93ad-f74dfc31057a service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:49.284125 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:49.285013 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:49.285296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:49.285575 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] PATH_INFO: `/v3/users/a8a32f1293714517b3c95b02d993d211` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:49.287034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a32f1293714517b3c95b02d993d211)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:49.288995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:49.512979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a32f1293714517b3c95b02d993d211'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.513765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a32f1293714517b3c95b02d993d211'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.525953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a32f1293714517b3c95b02d993d211'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.526614 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a32f1293714517b3c95b02d993d211'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.542811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-d3ddad0c-9c38-4e9c-a5d4-c7efbf68fbdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a32f1293714517b3c95b02d993d211'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:49.715028 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1383/2809] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:54:49 2022] DELETE /identity/v3/users/a8a32f1293714517b3c95b02d993d211 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:50.191374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:50.193112 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.209400 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f417133-cc88-464c-8c13-97ae59e8402f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:50.274078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:50.275039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.275375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.275793 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] PATH_INFO: `/v3/projects/ed877f67d66349bbb2244c4af4a2e214` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.277195 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed877f67d66349bbb2244c4af4a2e214)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:50.280817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:50.390337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed877f67d66349bbb2244c4af4a2e214', 'request_id': 'req-9883c490-2a07-44e5-8e06-8018c2a26b33'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:50.491887 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed877f67d66349bbb2244c4af4a2e214', 'request_id': 'req-9883c490-2a07-44e5-8e06-8018c2a26b33'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:50.498481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed877f67d66349bbb2244c4af4a2e214', 'request_id': 'req-9883c490-2a07-44e5-8e06-8018c2a26b33'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:50.511287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] The token cache is being invalidate because project ed877f67d66349bbb2244c4af4a2e214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:54:50.511580 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:50.513055 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9883c490-2a07-44e5-8e06-8018c2a26b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:54:50.514559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1427/2810] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:54:50 2022] DELETE /identity/v3/projects/ed877f67d66349bbb2244c4af4a2e214 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:50.519528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.519995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.520092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.540430 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:50.541739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.601833 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1384/2811] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:50.610835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.611125 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.611403 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.638421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:50.639494 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:50.651036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1428/2812] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:50.665166 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:50.676610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d4327-bf74-4aeb-a53d-c483634d0a7c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:50.682213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:50.683005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.683353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.683698 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.684495 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:54:50.684867 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:50.687225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:50.700871 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1385/2813] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:54:50.705462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:50.719694 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdc0ba-ec2c-4c37-b8bb-233afabe3d81 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:50.724618 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:50.725436 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.725734 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.725994 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.726737 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:50.728637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:50.848421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1429/2814] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:50 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:50.853600 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:50.864865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6561d0b-34eb-445e-8f05-8be9083f5907 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:50.929357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:50.929956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:50.930293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:50.930588 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:50.931289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:50.933258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:50.993702 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1386/2815] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:50 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:50.999192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.010491 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34229872-8a2c-43e5-ab34-544b5c42fd3b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.016858 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.017758 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.018043 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.019346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.019346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.021820 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.034286 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1430/2816] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:51.038400 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.050046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb8c0-df7d-46f1-90aa-aca9bac18b75 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.054737 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.055441 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.055725 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.056003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.056729 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.058415 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6ba32-d74f-4f91-982b-0a2422476b67 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.069511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1387/2817] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:51.074074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.084369 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f663-83a4-41c2-90e7-fdbec5252452 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.089268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.090274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.090561 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.090838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] PATH_INFO: `/v3/projects/8ff332ea67e34d74aba056c44eb9c1bf/users/cb679bc058ef491b8f5f1950a682aabf/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.105324 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff332ea67e34d74aba056c44eb9c1bf, user_id=cb679bc058ef491b8f5f1950a682aabf, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.109344 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.195628 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1431/2818] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:51 2022] PUT /identity/v3/projects/8ff332ea67e34d74aba056c44eb9c1bf/users/cb679bc058ef491b8f5f1950a682aabf/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:54:51.206134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.206623 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.206944 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.238638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] MFA Rules not processed for user `cb679bc058ef491b8f5f1950a682aabf`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:51.239991 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:51.314245 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1388/2819] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:51.318375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.392706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7d-2c76-416f-9242-479c3b3d9503 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.397302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.397956 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.398296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.398572 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.399295 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.401179 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.471502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1432/2820] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:54:51 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:54:51.475771 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.487444 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340ebf7-6368-4d92-b0ab-73c683c65fe0 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.545765 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.546491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.546778 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.547055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.547784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.549718 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.635100 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1389/2821] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:54:51 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:54:51.639860 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.651060 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac2e66-8f72-4674-852a-2c557b4d0cfa admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.657135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.657832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.658124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.658437 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.659224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.660859 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.677155 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1433/2822] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:51.681782 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.693027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ca59-1576-4121-911b-0ec0a4362738 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.697672 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.698604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.698883 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.699155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] PATH_INFO: `/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d/users/290493ef4bc64da8959b85a711dc0b2c/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.714409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6999cc0ddeff44dbabcd10b113e8c61d, user_id=290493ef4bc64da8959b85a711dc0b2c, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.717714 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.806851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1390/2823] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:51 2022] PUT /identity/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d/users/290493ef4bc64da8959b85a711dc0b2c/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:51.811313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.821458 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39619ada-1731-489d-a3c3-b82679659069 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.884084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.884756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.885035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.885308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.886027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.888652 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ae4eb-b761-4f55-9870-e0911cec6e29 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.900762 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1434/2824] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:54:51.905849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:51.916377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e30b-4252-465b-96d8-6cb5b3f3d0ba admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:51.921688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:51.922797 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:51.923087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:51.923368 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] PATH_INFO: `/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d/users/290493ef4bc64da8959b85a711dc0b2c/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:51.925383 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6999cc0ddeff44dbabcd10b113e8c61d, user_id=290493ef4bc64da8959b85a711dc0b2c, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:51.928117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:51.938621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1391/2825] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:54:51 2022] PUT /identity/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d/users/290493ef4bc64da8959b85a711dc0b2c/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:54:52.000693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:52.000908 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:52.001192 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:52.031507 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] MFA Rules not processed for user `290493ef4bc64da8959b85a711dc0b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:54:52.033398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:52.107175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1435/2826] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:54:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:54:52.117767 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:52.119456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:54:52.178646 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67b288-41a3-44f5-8bc2-89736daef548 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:52.182288 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:52.182941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:52.183223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:52.184105 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:52.193661 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:52.195155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:52.206247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1392/2827] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:54:52.310346 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:54:52.321609 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3857f-4791-4b7a-8140-39c23eb813bc None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:54:52.325819 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:54:52.326495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:54:52.326773 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:54:52.327049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:54:52.385289 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:54:52.386832 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0635-7964-4902-8f41-951f28f9ebcc service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:54:52.396918 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1436/2828] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:54:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:00.755784 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:00.821439 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a65b1b-7053-41f0-ad16-25e665b8c284 service cinder] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:00.828197 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:00.830687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:00.830687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:00.830687 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] PATH_INFO: `/v3/users/cb679bc058ef491b8f5f1950a682aabf` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:00.832393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb679bc058ef491b8f5f1950a682aabf)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:00.833596 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:01.032440 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb679bc058ef491b8f5f1950a682aabf'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:01.033117 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb679bc058ef491b8f5f1950a682aabf'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:01.047334 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb679bc058ef491b8f5f1950a682aabf'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:01.048086 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb679bc058ef491b8f5f1950a682aabf'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:01.057485 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb679bc058ef491b8f5f1950a682aabf'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:01.144902 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1393/2829] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:00 2022] DELETE /identity/v3/users/cb679bc058ef491b8f5f1950a682aabf => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:01.149531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:01.149992 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:01.150093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:01.169165 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:01.170444 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:01.241569 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1437/2830] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:01.252959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:01.254422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:01.314682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b2317-3b25-4a75-b2a2-21f50fc8ab99 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:01.318492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:01.319144 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:01.319429 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:01.319706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:01.329823 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:01.331198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:01.342142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1394/2831] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:01.797661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:01.799128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:01.820886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ecf8d-66d4-4b90-981f-d7b48f96f8d0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:01.826717 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:01.827537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:01.827814 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:01.828081 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] PATH_INFO: `/v3/projects/8ff332ea67e34d74aba056c44eb9c1bf` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:01.829296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff332ea67e34d74aba056c44eb9c1bf)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:01.831308 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:01.968158 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff332ea67e34d74aba056c44eb9c1bf', 'request_id': 'req-f08f2f12-d65a-4950-90a3-9b28175fef4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.040238 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff332ea67e34d74aba056c44eb9c1bf', 'request_id': 'req-f08f2f12-d65a-4950-90a3-9b28175fef4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.046568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff332ea67e34d74aba056c44eb9c1bf', 'request_id': 'req-f08f2f12-d65a-4950-90a3-9b28175fef4d'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.059933 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] The token cache is being invalidate because project 8ff332ea67e34d74aba056c44eb9c1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:02.060208 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.060876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.062661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1438/2832] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:01 2022] DELETE /identity/v3/projects/8ff332ea67e34d74aba056c44eb9c1bf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:02.066785 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:02.078364 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f49edf-cfc3-4601-806c-205bf30facd8 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:02.141860 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:02.142781 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:02.143069 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:02.143350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] PATH_INFO: `/v3/users/290493ef4bc64da8959b85a711dc0b2c` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:02.144530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=290493ef4bc64da8959b85a711dc0b2c)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:02.146496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:02.311473 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290493ef4bc64da8959b85a711dc0b2c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.312139 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290493ef4bc64da8959b85a711dc0b2c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.324621 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290493ef4bc64da8959b85a711dc0b2c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.325274 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290493ef4bc64da8959b85a711dc0b2c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.331916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-7a8bc9e6-df92-42cf-aba2-ba1ad9bdac8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290493ef4bc64da8959b85a711dc0b2c'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:02.423594 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1395/2833] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:02 2022] DELETE /identity/v3/users/290493ef4bc64da8959b85a711dc0b2c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:02.863359 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:02.865363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:02.879900 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2f12-d65a-4950-90a3-9b28175fef4d admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:02.946224 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:02.947279 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:02.947557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:02.947849 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] PATH_INFO: `/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:02.949487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6999cc0ddeff44dbabcd10b113e8c61d)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:02.951661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.053226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6999cc0ddeff44dbabcd10b113e8c61d', 'request_id': 'req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:03.137415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6999cc0ddeff44dbabcd10b113e8c61d', 'request_id': 'req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:03.143631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6999cc0ddeff44dbabcd10b113e8c61d', 'request_id': 'req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:03.156365 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] The token cache is being invalidate because project 6999cc0ddeff44dbabcd10b113e8c61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:03.156654 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:03.157326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-9fd57e70-5591-4d79-9f76-8d03fd5e32e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:03.158962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1439/2834] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:02 2022] DELETE /identity/v3/projects/6999cc0ddeff44dbabcd10b113e8c61d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:03.164901 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.165359 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.165481 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.186401 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:03.187633 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:03.250615 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1396/2835] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:03.258646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.258939 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.259241 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.278457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:03.279665 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:03.288218 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1440/2836] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:03.293293 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.302913 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf040-b1b3-4dc3-87b9-f83963b24521 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.308053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.308870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.309158 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.309442 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.310281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:55:03.310595 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.312233 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.326880 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1397/2837] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:55:03.328571 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.338008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206245-b084-40c0-b3e2-4c96da7d40d2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.343294 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.343945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.344235 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.344511 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.345212 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.347166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.442121 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1441/2838] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:03 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:03.446292 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.460509 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196fd5-a686-4721-ae21-0a4730bf643e admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.525849 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.527289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.527694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.527979 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.528706 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.530828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.617922 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1398/2839] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:03 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:03.622755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.633711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914de8-5099-4f0a-8faa-7313c90d9ce0 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.640672 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.641329 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.641610 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.641884 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.642637 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.644254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.656013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1442/2840] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:03.660102 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.670808 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe42ec-e2ee-4c0f-8fd3-a91597a5d311 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.677548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.678253 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.678543 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.678943 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.679678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.681858 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a504ac-6794-46ee-b36a-063c8ead0a3d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.695488 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1399/2841] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:03.700237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.710353 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64874d-551a-4053-9607-7604b16f6573 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:03.715853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:03.720959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.720959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.720959 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] PATH_INFO: `/v3/projects/07fa1be32bc84195b933e6745f7c5a21/users/35fab2727cb04708ad862da29850532d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.736223 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fa1be32bc84195b933e6745f7c5a21, user_id=35fab2727cb04708ad862da29850532d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:03.738932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:03.817091 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1443/2842] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:03 2022] PUT /identity/v3/projects/07fa1be32bc84195b933e6745f7c5a21/users/35fab2727cb04708ad862da29850532d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:03.830780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:03.831077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:03.831356 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:03.860726 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] MFA Rules not processed for user `35fab2727cb04708ad862da29850532d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:03.862002 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:03.930589 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1400/2843] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:03.943022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:03.944700 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:04.005313 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d328-6cdb-4c87-bcd2-13c62b6b57fa admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:04.009699 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:04.010380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:04.010664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:04.010942 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:04.020317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:04.021745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a85c0f-3ca4-450a-b125-22ed184d2525 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:04.030770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1444/2844] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:04.055783 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:04.123168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f7467-a945-4146-a98b-f01a660c0cd3 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:04.128221 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:04.129008 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:04.129296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:04.129616 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] PATH_INFO: `/v3/users/35fab2727cb04708ad862da29850532d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:04.130963 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35fab2727cb04708ad862da29850532d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:04.132946 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:04.335175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35fab2727cb04708ad862da29850532d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:04.336107 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35fab2727cb04708ad862da29850532d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:04.350313 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35fab2727cb04708ad862da29850532d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:04.351077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35fab2727cb04708ad862da29850532d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:04.359231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35fab2727cb04708ad862da29850532d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:04.455914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1401/2845] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:04 2022] DELETE /identity/v3/users/35fab2727cb04708ad862da29850532d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:04.460621 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:04.460915 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:04.461196 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:04.479501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:04.481522 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:04.546380 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1445/2846] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:04.556385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:04.558178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:04.628343 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e3b04-4597-4f30-bbb3-9b4bd74d9d5d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:04.632593 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:04.633249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:04.633528 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:04.633800 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:04.646231 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:04.647680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:04.659653 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1402/2847] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:04.983582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:04.985079 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:04.997193 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df626f-ec5c-4926-b508-94a7e2f5cdb2 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.017074 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.018633 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.018966 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.019259 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] PATH_INFO: `/v3/users/e620365b2a2948e291015e81b41bd3c1` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.020536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e620365b2a2948e291015e81b41bd3c1)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.022549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.153043 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.154791 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:05.169720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c4c03-9d75-4121-8a8e-bda9b8c362d5 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.177941 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.178896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.179161 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.179449 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] PATH_INFO: `/v3/projects/07fa1be32bc84195b933e6745f7c5a21` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.183684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fa1be32bc84195b933e6745f7c5a21)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.183684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.383971 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e620365b2a2948e291015e81b41bd3c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.384178 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fa1be32bc84195b933e6745f7c5a21', 'request_id': 'req-76ca64a0-e9b3-48b8-8a16-060dba81b31b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.384617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e620365b2a2948e291015e81b41bd3c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.397343 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e620365b2a2948e291015e81b41bd3c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.404646 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e620365b2a2948e291015e81b41bd3c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.405314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-55919023-26bc-40fc-a13f-14d9079487c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e620365b2a2948e291015e81b41bd3c1'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.485985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fa1be32bc84195b933e6745f7c5a21', 'request_id': 'req-76ca64a0-e9b3-48b8-8a16-060dba81b31b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.492395 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fa1be32bc84195b933e6745f7c5a21', 'request_id': 'req-76ca64a0-e9b3-48b8-8a16-060dba81b31b'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.504404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] The token cache is being invalidate because project 07fa1be32bc84195b933e6745f7c5a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:05.504693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.505336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-76ca64a0-e9b3-48b8-8a16-060dba81b31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:05.507034 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1403/2848] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:05 2022] DELETE /identity/v3/projects/07fa1be32bc84195b933e6745f7c5a21 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:55:05.512534 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a75b5-13b4-4c08-afd9-0131827e6c84 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.512822 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a75b5-13b4-4c08-afd9-0131827e6c84 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.513099 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a75b5-13b4-4c08-afd9-0131827e6c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.532619 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-735a75b5-13b4-4c08-afd9-0131827e6c84 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:05.533816 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-735a75b5-13b4-4c08-afd9-0131827e6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:05.571656 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1446/2849] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:04 2022] DELETE /identity/v3/users/e620365b2a2948e291015e81b41bd3c1 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:05.576801 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.577097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.577418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.594224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1404/2850] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:05.595732 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:05.596938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:05.601684 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.601974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.602294 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.621618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:05.622894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:05.659547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1447/2851] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:05.666007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.667986 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:05.681912 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1405/2852] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:05.686890 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.696001 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4ef84-7d27-4131-88b7-91e4b0dbd872 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.700939 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.701597 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.701877 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.702149 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.703055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:55:05.703362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.704955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.717036 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1406/2853] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 10 14:55:05.721692 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.727957 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f75a3-6009-4fd2-9381-31b60f1b578a None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.732350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f00da-347f-4cc1-9b32-f7b852f036e5 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.732723 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.733375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.733655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.733931 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.737249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.737896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.738179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.738491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.739179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.741113 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.743231 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.744647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.755780 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1448/2854] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:05.830688 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1407/2855] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:05 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:55:05.835114 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.845756 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbfc8d-3a94-4f73-9a84-26f070b599c6 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:05.902755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:05.903439 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:05.903724 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:05.904005 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:05.904695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:05.906675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:05.976776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1449/2856] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:05 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:55:05.982471 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:05.994010 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a23d4b-4f6e-467e-a335-24836448f9b1 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.000345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.001116 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.001456 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.001831 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.002746 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.004545 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.016003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1408/2857] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:55:06.020246 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.030692 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e4db5-54fc-421e-afa6-d304db3a15bb admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.035391 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.036083 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.036410 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.036697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.037464 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.039135 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.050661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1450/2858] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:06.054889 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.068345 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7429b-0aaa-44c2-ae01-19cbb614157a admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.073122 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.074362 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.074642 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.074918 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] PATH_INFO: `/v3/projects/c6c64611f011421cb7fee4437de09d64/users/86c060a7d99b4d098a10af19fded306d/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.089933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c64611f011421cb7fee4437de09d64, user_id=86c060a7d99b4d098a10af19fded306d, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.092651 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc22b2b-bd43-4e2a-bb0a-d5828ffeead9 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.188182 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.191266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:06.213647 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1409/2859] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:06 2022] PUT /identity/v3/projects/c6c64611f011421cb7fee4437de09d64/users/86c060a7d99b4d098a10af19fded306d/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:06.216953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4b46-03b0-4bbd-90c7-3852aaf31061 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.230552 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.230855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.231561 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.259625 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] MFA Rules not processed for user `86c060a7d99b4d098a10af19fded306d`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:06.260879 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:06.288139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.288998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.289282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.289562 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] PATH_INFO: `/v3/projects/50cf973c340644c5aee5349945ec9434` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.290776 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cf973c340644c5aee5349945ec9434)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.293325 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.326081 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1410/2860] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:06.337129 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.338496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:06.393415 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cf973c340644c5aee5349945ec9434', 'request_id': 'req-d907a154-842c-47b1-9421-fbaae23b1c09'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:06.396136 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d216474-6e0e-447e-aa08-121f6cde21a8 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.444894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.445467 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.445754 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.446042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.509406 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.511617 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bb493-1e5a-4af6-b5ef-7321a117fb4f service cinder] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.513530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cf973c340644c5aee5349945ec9434', 'request_id': 'req-d907a154-842c-47b1-9421-fbaae23b1c09'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:06.520594 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cf973c340644c5aee5349945ec9434', 'request_id': 'req-d907a154-842c-47b1-9421-fbaae23b1c09'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:06.522832 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1411/2861] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:06.533591 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] The token cache is being invalidate because project 50cf973c340644c5aee5349945ec9434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:06.535258 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:06.535949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-d907a154-842c-47b1-9421-fbaae23b1c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:06.537546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1451/2862] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:06 2022] DELETE /identity/v3/projects/50cf973c340644c5aee5349945ec9434 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:06.543643 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.543934 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.544217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.562788 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:06.563990 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:06.627165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1412/2863] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:06.634886 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.635173 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.635450 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.654838 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:06.656036 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:06.664002 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1452/2864] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:06.669040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.687047 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaa020-b179-4c59-a0a7-323ac33124f7 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.692222 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.692896 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.693176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.693451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.694224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:55:06.694566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.696215 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.710894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1413/2865] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:55:06.716440 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.726163 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584ebd-ad5d-4a64-978a-0457a0754755 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.731995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.732744 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.733101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.733453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.734250 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.736265 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:06.898938 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1453/2866] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:06 2022] POST /identity/v3/projects => generated 399 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:55:06.903657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:06.913811 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701d385-705b-4917-8721-b6d52a60c023 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:06.969908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:06.970660 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:06.970974 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:06.971249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:06.971969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:06.973895 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:07.091711 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1414/2867] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:06 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:07.095811 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:07.107381 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70173-3fb5-4deb-b492-d4b5b0e4d55b admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:07.112414 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:07.113255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:07.113389 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:07.113668 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:07.114559 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:07.116225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:07.130595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1454/2868] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:07.132140 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:07.148719 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695c09-367f-4efa-9bdd-7bca9287db14 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:07.150103 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:07.150801 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:07.151093 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:07.151404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:07.152239 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:07.155247 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce85de-93c8-4166-944a-eb2fe5795dc4 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:07.165339 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1415/2869] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 10 14:55:07.170038 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:07.180340 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08555a-ac1f-49c4-9102-b0427037d764 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:07.187144 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:07.188049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:07.188337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:07.188619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] PATH_INFO: `/v3/projects/8395e49f11794571b970779241c73a4b/users/7a8076481d814ade89547351fa6b3893/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:07.204418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8395e49f11794571b970779241c73a4b, user_id=7a8076481d814ade89547351fa6b3893, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:07.208932 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:07.285531 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1455/2870] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:07 2022] PUT /identity/v3/projects/8395e49f11794571b970779241c73a4b/users/7a8076481d814ade89547351fa6b3893/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:07.297566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:07.297854 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:07.298132 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:07.325236 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] MFA Rules not processed for user `7a8076481d814ade89547351fa6b3893`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:07.326480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:07.395106 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1416/2871] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:07.409564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:07.409564 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:07.465015 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a243e0-b2a4-41af-a29e-400b67f5fd89 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:07.468619 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:07.469266 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:07.470089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:07.470089 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:07.478649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:07.480136 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70eca1-ef99-4b5a-9ea1-c44a896fc4c5 service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:07.489953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1456/2872] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:20.661354 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:20.745547 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ae17b-8049-435b-9cf3-b609ab7cfbc2 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:20.752955 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:20.754304 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:20.754601 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:20.754866 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] PATH_INFO: `/v3/users/7a8076481d814ade89547351fa6b3893` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:20.756115 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8076481d814ade89547351fa6b3893)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:20.759271 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:20.963352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8076481d814ade89547351fa6b3893'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:20.964367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8076481d814ade89547351fa6b3893'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:20.985381 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8076481d814ade89547351fa6b3893'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:20.986164 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8076481d814ade89547351fa6b3893'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:20.993694 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8076481d814ade89547351fa6b3893'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.091399 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1417/2873] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:20 2022] DELETE /identity/v3/users/7a8076481d814ade89547351fa6b3893 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:21.096500 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:21.097007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:21.097398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:21.124530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:21.126693 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:21.192806 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1457/2874] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:21.202530 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:21.203953 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:21.264262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027be9e9-ff23-4a4d-b21b-09e9a4477cb3 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:21.269496 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:21.270659 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:21.270882 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:21.271199 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:21.283262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:21.284712 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:21.296250 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1418/2875] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:21.749107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:21.751418 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:21.764589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e24ac-0cb8-4e5c-9384-39df01e9494c None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:21.769895 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:21.770885 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:21.771244 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:21.771585 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] PATH_INFO: `/v3/projects/8395e49f11794571b970779241c73a4b` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:21.772896 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8395e49f11794571b970779241c73a4b)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:21.775395 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:21.884975 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8395e49f11794571b970779241c73a4b', 'request_id': 'req-863044d7-938d-4d1e-85c1-173304b1062f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.969053 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8395e49f11794571b970779241c73a4b', 'request_id': 'req-863044d7-938d-4d1e-85c1-173304b1062f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.975049 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8395e49f11794571b970779241c73a4b', 'request_id': 'req-863044d7-938d-4d1e-85c1-173304b1062f'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.987831 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] The token cache is being invalidate because project 8395e49f11794571b970779241c73a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:21.988124 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.988786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-863044d7-938d-4d1e-85c1-173304b1062f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:21.990401 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1458/2876] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:21 2022] DELETE /identity/v3/projects/8395e49f11794571b970779241c73a4b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:25.905891 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:25.907420 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:25.921336 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce9623-17f0-4142-8ea4-013ce44ab7e6 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:25.983855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:25.984738 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:25.985025 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:25.985306 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] PATH_INFO: `/v3/users/86c060a7d99b4d098a10af19fded306d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:25.986553 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c060a7d99b4d098a10af19fded306d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:25.988497 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:26.173964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c060a7d99b4d098a10af19fded306d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:26.174680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c060a7d99b4d098a10af19fded306d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:26.188213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c060a7d99b4d098a10af19fded306d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:26.188933 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c060a7d99b4d098a10af19fded306d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:26.195749 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c060a7d99b4d098a10af19fded306d'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:26.289316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1419/2877] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:25 2022] DELETE /identity/v3/users/86c060a7d99b4d098a10af19fded306d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:26.295023 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.295502 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.295588 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.314539 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:26.315742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.378864 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1459/2878] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:26.388667 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:26.390110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.449363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e098-61e8-42ea-896e-f1ba4a3648c2 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:26.453148 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:26.453845 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.454123 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.458959 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.464697 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:26.466191 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70eee6-e2eb-46e7-9467-ac9939f24cd9 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:26.476510 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1420/2879] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:26.937973 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:26.942337 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:26.951626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da9a49-d106-4dae-9b36-0c1405280997 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:26.956471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:26.957273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:26.957645 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:26.958057 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] PATH_INFO: `/v3/projects/c6c64611f011421cb7fee4437de09d64` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:26.959530 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c64611f011421cb7fee4437de09d64)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:26.961477 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.155473 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c64611f011421cb7fee4437de09d64', 'request_id': 'req-b11521fc-30af-4bc3-953f-6c372fffe3ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:27.244834 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c64611f011421cb7fee4437de09d64', 'request_id': 'req-b11521fc-30af-4bc3-953f-6c372fffe3ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:27.251303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c64611f011421cb7fee4437de09d64', 'request_id': 'req-b11521fc-30af-4bc3-953f-6c372fffe3ed'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:27.263117 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] The token cache is being invalidate because project c6c64611f011421cb7fee4437de09d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:27.263402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:27.264065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-b11521fc-30af-4bc3-953f-6c372fffe3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:27.265664 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1460/2880] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:26 2022] DELETE /identity/v3/projects/c6c64611f011421cb7fee4437de09d64 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:27.270956 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.271431 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.271517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.290011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:27.291357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:27.353057 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1421/2881] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:27.362449 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d9549-0226-4013-8911-5b114d555515 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.363293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d9549-0226-4013-8911-5b114d555515 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.363293 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d9549-0226-4013-8911-5b114d555515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.381427 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-c44d9549-0226-4013-8911-5b114d555515 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:27.382661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c44d9549-0226-4013-8911-5b114d555515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:27.391790 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1461/2882] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:27.396243 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.406855 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0ca59-2ae9-4a24-b2e3-fa5353a13a63 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.411486 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.412142 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.412426 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.412703 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.413422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:55:27.413733 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.415377 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.427305 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1422/2883] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:55:27.434141 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d9549-0226-4013-8911-5b114d555515 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.443719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d9549-0226-4013-8911-5b114d555515 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.449281 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.449945 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.450267 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.450548 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.451254 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.453178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.589655 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1462/2884] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:27 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:27.593973 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.604088 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602f2bc-8a07-4665-bcb1-ab826ce15642 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.659346 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.660029 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.660296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.660568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.661257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.663202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.765761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1423/2885] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:27 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:27.770448 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.780890 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72239454-5dcc-40f4-b81a-ea24a0b07114 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.785469 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.786326 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.786607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.787028 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.787760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.789362 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.801471 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1463/2886] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:27.807761 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.818059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bbec7-5cfd-4ba0-bd89-c4466606396d admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.823763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.824490 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.824777 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.825055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.825837 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.827527 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935847c0-a705-4307-bf6e-b72e7d7954c7 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.840104 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1424/2887] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:27.844457 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:27.856303 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6002-7575-4318-bc44-66099d070aa1 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:27.861106 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:27.862039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.862348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.862626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] PATH_INFO: `/v3/projects/4f78da2d8c0d41e1826839888feed24e/users/9743c6c4397e496fb77fe02cabbe4fd3/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.878789 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f78da2d8c0d41e1826839888feed24e, user_id=9743c6c4397e496fb77fe02cabbe4fd3, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:27.881459 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:27.958582 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1464/2888] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:27 2022] PUT /identity/v3/projects/4f78da2d8c0d41e1826839888feed24e/users/9743c6c4397e496fb77fe02cabbe4fd3/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:27.969179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:27.969466 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:27.969747 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:27.998030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] MFA Rules not processed for user `9743c6c4397e496fb77fe02cabbe4fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:27.999263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.061312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1425/2889] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:28.072274 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:28.074084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.130305 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d34938-f644-4bf9-9c4c-de5e6c6439e3 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:28.133993 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef044e8d1fb4953b2cc3244f4a4369f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:28.135711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.135711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.135711 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.144014 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:28.145480 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794f9ba-8935-4061-95fe-de96b82b2fbb service cinder] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:28.154399 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1465/2890] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:28.257885 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:28.259282 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.324500 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179a402-4308-44a2-bf82-96941b8dd883 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:28.329457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:28.330373 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.330657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.330927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] PATH_INFO: `/v3/users/ad45aac8279d45209d3b4e3bc88fb729` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.332151 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad45aac8279d45209d3b4e3bc88fb729)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:28.334181 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:28.571894 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad45aac8279d45209d3b4e3bc88fb729'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:28.572605 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad45aac8279d45209d3b4e3bc88fb729'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:28.584577 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad45aac8279d45209d3b4e3bc88fb729'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:28.585257 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad45aac8279d45209d3b4e3bc88fb729'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:28.593645 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad45aac8279d45209d3b4e3bc88fb729'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:28.673599 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1426/2891] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:28 2022] DELETE /identity/v3/users/ad45aac8279d45209d3b4e3bc88fb729 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:28.678226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.678698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.678949 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.697792 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:28.699093 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.760730 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1466/2892] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:28.770678 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:28.772087 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:28.835355 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3144a6-ee7c-4e75-94d0-7a24fd2cd681 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:28.839016 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:28.839648 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:28.839927 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:28.840201 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:28.849836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:28.851289 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22997d0-420d-40fc-a609-52d23cb07c01 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:28.861814 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1427/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:29.273698 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:29.275317 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:29.287796 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d952a73-f6d3-4203-b25e-7e509de558b4 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:29.292968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:29.293833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.294113 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.294422 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] PATH_INFO: `/v3/projects/efd6fab5c21d4140a5bc407e035999bd` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.295620 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd6fab5c21d4140a5bc407e035999bd)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:29.297644 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:29.467499 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd6fab5c21d4140a5bc407e035999bd', 'request_id': 'req-24de607f-04af-4381-9bf4-4ed4a4990c5a'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:29.667967 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd6fab5c21d4140a5bc407e035999bd', 'request_id': 'req-24de607f-04af-4381-9bf4-4ed4a4990c5a'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:29.674216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd6fab5c21d4140a5bc407e035999bd', 'request_id': 'req-24de607f-04af-4381-9bf4-4ed4a4990c5a'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:29.686396 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] The token cache is being invalidate because project efd6fab5c21d4140a5bc407e035999bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:29.687008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:29.687487 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-24de607f-04af-4381-9bf4-4ed4a4990c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:29.689066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1467/2894] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:29 2022] DELETE /identity/v3/projects/efd6fab5c21d4140a5bc407e035999bd => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:55:29.694077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.694526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.694717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.712421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:29.714332 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:29.774165 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1428/2895] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:29.782865 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.783154 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.783430 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.802568 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:29.803759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:29.811804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1468/2896] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:29.816004 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:29.825683 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eeffed-f84a-45ed-8ef8-026a331da3a1 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:29.830341 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:29.830998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.831275 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.831548 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] PATH_INFO: `/v3/domains` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.832297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80413) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 10 14:55:29.832604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:29.834177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1473f-62ae-4683-a36d-c7adc90ac11b admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:29.846097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1429/2897] 10.222.0.58 () {58 vars in 1152 bytes} [Tue May 10 14:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 10 14:55:29.852987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:29.863661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44f48e-90b6-484a-9573-bdff73b3a718 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:29.868563 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:29.869815 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.874216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.874216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.874216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:29.874216 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:29.946522 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:29.946828 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:29.947217 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:29.965434 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] MFA Rules not processed for user `9743c6c4397e496fb77fe02cabbe4fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:29.966665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:29.972156 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1469/2898] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:29 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:29.981242 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.021995 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1430/2899] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:30.044210 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f9902-ecd9-4e74-9106-e5e67efd9365 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.049109 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.051852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.051852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.051852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] PATH_INFO: `/v3/users` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.051852 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.052995 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.143394 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1470/2900] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:29 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:55:30.147690 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.159506 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07911-8fc5-4115-9de3-a8e1e0016641 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.164205 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.164930 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.165213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.165492 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.166454 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.168071 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.178511 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1431/2901] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:30.182635 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.192875 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbff615-2ed7-4424-8f5c-6f3239c5843f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.199268 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.199944 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.200349 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.200643 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.201388 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.203034 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307e74c-a4cb-4d4d-aab8-958539eb3914 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.214962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1471/2902] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:30.218766 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.229673 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda5111-46b8-4d10-83fd-cfc4ab41be50 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.243350 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.244348 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.244720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.244908 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] PATH_INFO: `/v3/projects/7ad02346a7e24c6ca4b182bb65090916/users/edfc0d3911964db6b35768373ddb8237/roles/88a0e9d0fe9d48eaae419a9d79677539` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.263082 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad02346a7e24c6ca4b182bb65090916, user_id=edfc0d3911964db6b35768373ddb8237, role_id=88a0e9d0fe9d48eaae419a9d79677539)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.265851 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.356297 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1432/2903] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:30 2022] PUT /identity/v3/projects/7ad02346a7e24c6ca4b182bb65090916/users/edfc0d3911964db6b35768373ddb8237/roles/88a0e9d0fe9d48eaae419a9d79677539 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:55:30.361714 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.362237 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.362416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.381178 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:30.382479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:30.445817 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1472/2904] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:30.460836 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.463108 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:30.522098 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7502c-1766-4540-a500-eb39bddd2cd0 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.526030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.526720 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.527011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.527290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.536568 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.538003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.548360 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1433/2905] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:30.652661 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:30.654020 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:30.734803 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7ed2a-d271-48b5-a445-21654a298804 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:30.740146 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:30.740804 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:30.741101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:30.741366 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:30.743226 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:30.755936 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:30.757537 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:30.768755 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1473/2906] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 10 14:55:35.528130 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:35.530085 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:35.552353 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d76-7a7f-4da5-8357-a02054c3a41c service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:35.558717 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:35.559938 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:35.560460 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:35.561030 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:35.563352 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:35.581842 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:35.585202 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bc8ef-4bc7-4939-9eac-70c45f919387 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:35.600003 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1434/2907] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:36.972402 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:36.974513 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:37.117958 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aaefb-1bfc-4ead-b72a-9e00288cc9f4 service placement] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:37.136862 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559efa42-6542-4923-9216-92f801743f61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:37.137874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559efa42-6542-4923-9216-92f801743f61 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:37.138276 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559efa42-6542-4923-9216-92f801743f61 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:37.138595 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559efa42-6542-4923-9216-92f801743f61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:37.140663 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-559efa42-6542-4923-9216-92f801743f61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:37.155166 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559efa42-6542-4923-9216-92f801743f61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:37.157675 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559efa42-6542-4923-9216-92f801743f61 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:37.178234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1474/2908] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:38.589676 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.590263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.590263 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.619763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] MFA Rules not processed for user `edfc0d3911964db6b35768373ddb8237`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:38.620998 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:38.691583 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1435/2909] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:38.697100 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559efa42-6542-4923-9216-92f801743f61 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:38.708201 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559efa42-6542-4923-9216-92f801743f61 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:38.713336 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:38.714084 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.714495 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.714983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] PATH_INFO: `/v3/projects` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.715745 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:38.722481 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:38.828706 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1475/2910] 10.222.0.58 () {60 vars in 1152 bytes} [Tue May 10 14:55:38 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 10 14:55:38.832935 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:38.844610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e7785-559e-4060-a963-e6ad5e0c1e54 None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:38.902988 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:38.903724 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:38.904006 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:38.904285 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] PATH_INFO: `/v3/users` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:38.904969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:38.906914 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:39.004691 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1436/2911] 10.222.0.58 () {60 vars in 1143 bytes} [Tue May 10 14:55:38 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 10 14:55:39.008902 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:39.020453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92dea-de19-4b8c-b26d-be95c208704f admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:39.027039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:39.027725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:39.028008 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:39.028282 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:39.029026 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:39.030742 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:39.042768 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1476/2912] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:39.047240 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:39.058518 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf21d87-cb9e-4f85-b2e7-c157c19205ab admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:39.065322 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:39.066296 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:39.066584 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:39.066865 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] PATH_INFO: `/v3/projects/6c0b09ddfb1b412199b168462e4a988d/users/a387d9efe11449d8ba8da9f5fffc2d79/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:39.084325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0b09ddfb1b412199b168462e4a988d, user_id=a387d9efe11449d8ba8da9f5fffc2d79, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:39.087629 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:39.160169 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1437/2913] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:39 2022] PUT /identity/v3/projects/6c0b09ddfb1b412199b168462e4a988d/users/a387d9efe11449d8ba8da9f5fffc2d79/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 14:55:39.164398 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:39.179987 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540ca46-6043-42d9-a9b3-7011291a12b7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:39.247022 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:39.247736 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:39.248000 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:39.248275 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:39.249363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:39.250658 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:39.262546 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1477/2914] 10.222.0.58 () {58 vars in 1121 bytes} [Tue May 10 14:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 10 14:55:39.266679 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:39.278258 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b893c-7790-4060-9b8a-90a02a6a1798 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:39.283077 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:39.283978 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:39.284266 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:39.284541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] PATH_INFO: `/v3/projects/6c0b09ddfb1b412199b168462e4a988d/users/a387d9efe11449d8ba8da9f5fffc2d79/roles/1c3f38f1141a421ea1f2f6dd9bff328f` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:39.286610 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0b09ddfb1b412199b168462e4a988d, user_id=a387d9efe11449d8ba8da9f5fffc2d79, role_id=1c3f38f1141a421ea1f2f6dd9bff328f)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:39.289290 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d0d5c-e98b-425c-ae69-da73b4b59b93 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:39.300517 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1438/2915] 10.222.0.58 () {60 vars in 1482 bytes} [Tue May 10 14:55:39 2022] PUT /identity/v3/projects/6c0b09ddfb1b412199b168462e4a988d/users/a387d9efe11449d8ba8da9f5fffc2d79/roles/1c3f38f1141a421ea1f2f6dd9bff328f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:55:44.660702 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:44.664519 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:44.779252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4553936-c4b4-4518-a614-3480e9c819ef admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:44.865925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:44.866626 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:44.866909 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:44.867185 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:44.869704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:44.884150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:44.885810 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:44.899252 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1478/2916] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:46.567283 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a552-22c8-4475-b862-fef1baa6b392 None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:46.567790 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a552-22c8-4475-b862-fef1baa6b392 None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:46.567985 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653a552-22c8-4475-b862-fef1baa6b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:46.597421 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-5653a552-22c8-4475-b862-fef1baa6b392 None None] MFA Rules not processed for user `a387d9efe11449d8ba8da9f5fffc2d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:46.598739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-5653a552-22c8-4475-b862-fef1baa6b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:46.695520 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1439/2917] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:46.710544 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:46.729833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99752e1-f2c0-4974-b1cf-254757772f0c service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:46.736098 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:46.736754 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:46.737039 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:46.737314 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:46.825027 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:46.828035 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f15f-068d-47f2-ad07-37e140791302 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:46.846447 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1479/2918] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:50.137055 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:50.137541 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:50.137624 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:50.167382 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] MFA Rules not processed for user `edfc0d3911964db6b35768373ddb8237`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:50.168618 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:50.201396 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1440/2919] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:52.408321 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0994c-8473-4dd8-a794-6bc0babd3750 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:52.408827 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0994c-8473-4dd8-a794-6bc0babd3750 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:52.408926 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0994c-8473-4dd8-a794-6bc0babd3750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:52.473416 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-36e0994c-8473-4dd8-a794-6bc0babd3750 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:52.475110 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-36e0994c-8473-4dd8-a794-6bc0babd3750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:52.549175 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1480/2920] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:52.706056 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:52.707622 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:52.807286 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f7fe2-72c4-464f-ab91-22047267c5bf None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:52.814320 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:52.814967 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:52.815249 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:52.815525 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:52.829269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:52.831728 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:52.841835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1441/2921] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:55:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:55:53.725738 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397920d-5e7b-459c-98ad-968bd9b24e65 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:53.726153 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397920d-5e7b-459c-98ad-968bd9b24e65 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:53.726322 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397920d-5e7b-459c-98ad-968bd9b24e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:53.744968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4397920d-5e7b-459c-98ad-968bd9b24e65 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:53.746139 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4397920d-5e7b-459c-98ad-968bd9b24e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:53.753334 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1481/2922] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 10 14:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:53.767899 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:53.779027 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e886dd-6d71-4147-831c-1cc87d15a975 service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:53.791870 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:53.792657 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:53.793262 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:53.793835 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:53.808857 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:53.816962 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:53.818764 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1442/2923] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:55:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:55:54.547419 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00922c-7791-45f3-a05f-6c8f96153a5f None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:54.548061 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00922c-7791-45f3-a05f-6c8f96153a5f None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:54.548376 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00922c-7791-45f3-a05f-6c8f96153a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:54.569101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-4a00922c-7791-45f3-a05f-6c8f96153a5f None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:54.569101 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-4a00922c-7791-45f3-a05f-6c8f96153a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:54.576894 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1482/2924] 10.222.0.58 () {58 vars in 1009 bytes} [Tue May 10 14:55:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:54.589367 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:54.600514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e26997-bb3b-4046-a5ec-f880000fd9fa service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:54.606112 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:54.606863 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:54.607141 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:54.607453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:54.616089 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:54.618173 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:54.626693 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1443/2925] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 10 14:55:55.203397 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:55.204063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:55.204063 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:55.223013 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] MFA Rules not processed for user `5dfa457ceb974f0b9c5200c213188dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:55.224025 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:55.232512 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1483/2926] 10.222.0.58 () {58 vars in 967 bytes} [Tue May 10 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:55:55.245960 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:55.262281 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb68f-4443-49c6-ab83-2d6bb3c6f8f4 service swift] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:55.270546 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ef235951c84f62b91e30d56860a904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:55.271315 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:55.271480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:55.271780 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:55.282403 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:55.284076 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8af41-482b-4cb2-a9f5-fcdb29280dd2 service swift] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:55.297180 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1444/2927] 10.222.0.58 () {60 vars in 1384 bytes} [Tue May 10 14:55:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 10 14:55:58.134725 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:58.136479 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:58.202855 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb469ce-1ce6-4534-87f4-6eca145a6ab0 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:58.207786 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:58.208667 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:58.208953 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:58.209239 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] PATH_INFO: `/v3/users/9743c6c4397e496fb77fe02cabbe4fd3` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:58.210536 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9743c6c4397e496fb77fe02cabbe4fd3)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:58.212607 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:58.393128 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9743c6c4397e496fb77fe02cabbe4fd3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:58.393968 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9743c6c4397e496fb77fe02cabbe4fd3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:58.407876 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9743c6c4397e496fb77fe02cabbe4fd3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:58.420462 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9743c6c4397e496fb77fe02cabbe4fd3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:58.421612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9743c6c4397e496fb77fe02cabbe4fd3'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:58.561050 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1484/2928] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 14:55:58 2022] DELETE /identity/v3/users/9743c6c4397e496fb77fe02cabbe4fd3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:55:58.567514 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:58.568269 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:58.568498 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:58.589408 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] MFA Rules not processed for user `462840e7ac324edfaa49acd55da38f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:55:58.590680 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:58.657111 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1445/2929] 10.222.0.58 () {58 vars in 957 bytes} [Tue May 10 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:55:58.666874 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:58.668287 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:58.741766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3e53-d366-4d28-94b1-529e105dca28 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:58.747550 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:58.748347 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:58.748639 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:58.749150 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:58.761107 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:58.762627 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:58.775808 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1485/2930] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:59.041932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:59.043549 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:59.062456 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:55:59.064600 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:59.091126 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968967be-2327-4a6c-88dd-5f04efa08c4b service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:59.106760 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:59.107753 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:59.108032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:59.108363 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] PATH_INFO: `/v3/projects/4f78da2d8c0d41e1826839888feed24e` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:59.109690 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f78da2d8c0d41e1826839888feed24e)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:59.112922 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:59.137964 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193143c-b066-4ad2-8236-655b907dc18c None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:55:59.143529 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:55:59.144213 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:55:59.144491 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:55:59.144772 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:55:59.146655 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:55:59.159916 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:55:59.161526 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:55:59.175127 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1446/2931] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:55:59.224775 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f78da2d8c0d41e1826839888feed24e', 'request_id': 'req-355b0fd1-e350-4347-8d8b-fff7f21217d7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:59.294428 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f78da2d8c0d41e1826839888feed24e', 'request_id': 'req-355b0fd1-e350-4347-8d8b-fff7f21217d7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:59.302719 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f78da2d8c0d41e1826839888feed24e', 'request_id': 'req-355b0fd1-e350-4347-8d8b-fff7f21217d7'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:59.318065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] The token cache is being invalidate because project 4f78da2d8c0d41e1826839888feed24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80414) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 14:55:59.318065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:59.318065 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 14:55:59.318484 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1486/2932] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 14:55:59 2022] DELETE /identity/v3/projects/4f78da2d8c0d41e1826839888feed24e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 14:56:25.251630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:25.253385 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:25.267735 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc32cf-ba97-447e-b94d-3996119d9e54 service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:25.318422 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:25.319888 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:25.320177 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:25.320453 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:25.322338 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:25.331634 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:25.333046 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:25.341742 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1447/2933] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:56:47.663007 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:56:47.664420 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:47.679018 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0fd1-e350-4347-8d8b-fff7f21217d7 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:56:47.682636 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:56:47.683273 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:56:47.683557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:56:47.683833 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:56:47.685529 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:56:47.749615 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:56:47.751072 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:56:47.760547 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1487/2934] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:56:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:59:04.168179 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:59:04.178278 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5353-3922-40b7-8ce2-6db1790b6a76 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:59:04.182168 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:59:04.183040 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:04.183176 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:04.183417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:04.185225 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:59:04.261325 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:59:04.262932 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a285b6-29a4-4514-aca8-f686430da594 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:59:04.274092 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1448/2935] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:59:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:59:05.447514 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:59:05.456433 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54db0c-4b76-486e-9a5c-db340e84ea39 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:59:05.460052 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:59:05.460695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:05.460980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:05.461257 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:05.463070 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:59:05.476617 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:59:05.478032 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c473f-1f4e-4791-8ddc-25cd6145afe0 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:59:05.489785 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1488/2936] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:59:23.713838 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44addcb9-704c-4e89-82aa-8d3cd2abf56d None None] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:23.714366 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44addcb9-704c-4e89-82aa-8d3cd2abf56d None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:23.714461 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44addcb9-704c-4e89-82aa-8d3cd2abf56d None None] PATH_INFO: `/` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:23.715829 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1449/2937] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:59:23.726977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1554854-43dd-4f9c-8455-8412caeadf57 None None] REQUEST_METHOD: `POST` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:23.726977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1554854-43dd-4f9c-8455-8412caeadf57 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:23.726977 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1554854-43dd-4f9c-8455-8412caeadf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:23.787990 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.auth.core [None req-d1554854-43dd-4f9c-8455-8412caeadf57 None None] MFA Rules not processed for user `cf1f369c503e48b08e3faa9738c5df8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:59:23.789225 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-d1554854-43dd-4f9c-8455-8412caeadf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:59:23.795998 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1489/2938] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 10 14:59:23.809059 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44addcb9-704c-4e89-82aa-8d3cd2abf56d None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:59:23.820763 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44addcb9-704c-4e89-82aa-8d3cd2abf56d None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:59:23.824223 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:59:23.824853 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:23.825134 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:23.825409 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:23.832393 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:59:23.833825 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a804ea-142a-4b1a-9f17-9f1a1308fb30 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:59:23.841736 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1450/2939] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 14:59:23.923649 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d304-76d6-4074-889b-4956f20c79d6 None None] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:23.924147 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d304-76d6-4074-889b-4956f20c79d6 None None] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:23.924541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d304-76d6-4074-889b-4956f20c79d6 None None] PATH_INFO: `/` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:23.925925 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1490/2940] 10.222.0.58 () {54 vars in 921 bytes} [Tue May 10 14:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 10 14:59:23.930887 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] REQUEST_METHOD: `POST` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:23.931175 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:23.931451 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:23.968638 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.auth.core [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] MFA Rules not processed for user `ffdd6ee2e1e64d2fb0d6d05687df82c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80413) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 10 14:59:23.969852 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 14:59:23.979457 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1451/2941] 10.222.0.58 () {58 vars in 1018 bytes} [Tue May 10 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 10 14:59:23.990589 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d304-76d6-4074-889b-4956f20c79d6 None None] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 14:59:24.002704 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d304-76d6-4074-889b-4956f20c79d6 None None] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 14:59:24.006441 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 14:59:24.007078 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 14:59:24.007357 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 14:59:24.007631 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 14:59:24.018421 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 14:59:24.019853 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 14:59:24.030302 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1491/2942] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:05.039299 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:05.040682 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:05.149789 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fec2-d46c-4dd8-ab85-4e211787204f None None] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:05.154969 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:05.155665 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:05.155947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:05.156224 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:05.158053 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:05.170604 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:05.172759 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:05.184579 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1452/2943] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:01:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:09.742076 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:09.751659 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cb7c-2224-4f56-a91a-9a60c97d39ed service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:09.755374 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:09.756012 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:09.756296 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:09.756575 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:09.758375 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:09.768090 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:09.769510 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:09.777541 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1492/2944] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:11.414110 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:11.423931 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b8353-f48a-4f3b-8928-9026340c69e6 service placement] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:11.427567 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:11.428198 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:11.428480 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:11.428756 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:11.430531 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:11.489758 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:11.491118 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:11.500598 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1453/2945] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:01:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:12.492759 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:12.495009 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:12.515695 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87058237-5c3a-4e29-8ac6-097051b76ad1 service neutron] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:12.521835 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffdd6ee2e1e64d2fb0d6d05687df82c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:12.522857 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:12.523234 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:12.523729 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:12.525766 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:12.543407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:12.545557 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:12.558062 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1493/2946] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:17.909011 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:17.910947 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:17.925194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbefb4-a2bb-4b53-ae04-9b3b9f86f85e service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:17.929067 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf1f369c503e48b08e3faa9738c5df8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:17.929739 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:17.930017 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:17.930312 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:17.932042 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:18.020576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:18.022005 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:18.041417 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1454/2947] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 10 15:01:26.007962 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:26.009549 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:26.024941 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668be419-6c8a-456b-bbf8-87ecb7e06a99 service nova] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:26.029770 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:26.031177 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:26.031466 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:26.031743 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] PATH_INFO: `/v3/users/edfc0d3911964db6b35768373ddb8237` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:26.032970 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfc0d3911964db6b35768373ddb8237)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:26.034983 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:26.192407 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfc0d3911964db6b35768373ddb8237'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.193066 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfc0d3911964db6b35768373ddb8237'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.207648 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfc0d3911964db6b35768373ddb8237'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.216783 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfc0d3911964db6b35768373ddb8237'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.220757 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfc0d3911964db6b35768373ddb8237'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.304221 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1494/2948] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 15:01:26 2022] DELETE /identity/v3/users/edfc0d3911964db6b35768373ddb8237 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:01:26.570603 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:26.585537 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05894a-a9ed-42f6-bc09-0adb9f179ba8 service neutron] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:26.648045 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:26.649009 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:26.649287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:26.649630 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] PATH_INFO: `/v3/projects/7ad02346a7e24c6ca4b182bb65090916` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:26.650905 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad02346a7e24c6ca4b182bb65090916)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:26.652897 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:26.774316 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad02346a7e24c6ca4b182bb65090916', 'request_id': 'req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.864194 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad02346a7e24c6ca4b182bb65090916', 'request_id': 'req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.870576 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad02346a7e24c6ca4b182bb65090916', 'request_id': 'req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.884068 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] The token cache is being invalidate because project 7ad02346a7e24c6ca4b182bb65090916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:01:26.884404 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.896267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:26.896267 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1455/2949] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 15:01:26 2022] DELETE /identity/v3/projects/7ad02346a7e24c6ca4b182bb65090916 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:01:26.896535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:26.896535 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:26.906980 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852298-f723-4ef0-9ef4-5b268ae80d26 admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:26.971033 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:26.971897 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:26.972183 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:26.972453 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] PATH_INFO: `/v3/users/a387d9efe11449d8ba8da9f5fffc2d79` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:26.973647 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a387d9efe11449d8ba8da9f5fffc2d79)` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:26.975612 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:27.278361 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a387d9efe11449d8ba8da9f5fffc2d79'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.279030 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a387d9efe11449d8ba8da9f5fffc2d79'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.291405 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a387d9efe11449d8ba8da9f5fffc2d79'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.298256 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a387d9efe11449d8ba8da9f5fffc2d79'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.298940 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.notifications [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a387d9efe11449d8ba8da9f5fffc2d79'} {{(pid=80414) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.396351 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1495/2950] 10.222.0.58 () {58 vars in 1223 bytes} [Tue May 10 15:01:26 2022] DELETE /identity/v3/users/a387d9efe11449d8ba8da9f5fffc2d79 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 10 15:01:27.648637 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Authenticating user token {{(pid=80413) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:01:27.650048 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.fernet_utils [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80413) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:01:27.662984 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74ca5-104f-4bdc-85fa-c1df47dd84a3 admin admin] Validating token access rules against request {{(pid=80413) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:01:27.719566 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462840e7ac324edfaa49acd55da38f49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fab88b77b7fd44528a3ca6841d9f273b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80413) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:01:27.721128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:01:27.721128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] SCRIPT_NAME: `/identity` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:01:27.721128 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] PATH_INFO: `/v3/projects/6c0b09ddfb1b412199b168462e4a988d` {{(pid=80413) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:01:27.722155 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0b09ddfb1b412199b168462e4a988d)` {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:01:27.724097 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] RBAC: Authorization granted {{(pid=80413) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:01:27.836681 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0b09ddfb1b412199b168462e4a988d', 'request_id': 'req-67ba453c-fc27-4406-a246-30aa806c056e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.921585 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0b09ddfb1b412199b168462e4a988d', 'request_id': 'req-67ba453c-fc27-4406-a246-30aa806c056e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.927363 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0b09ddfb1b412199b168462e4a988d', 'request_id': 'req-67ba453c-fc27-4406-a246-30aa806c056e'} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.938357 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] The token cache is being invalidate because project 6c0b09ddfb1b412199b168462e4a988d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80413) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 10 15:01:27.938650 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.939287 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: DEBUG keystone.notifications [None req-67ba453c-fc27-4406-a246-30aa806c056e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80413) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 10 15:01:27.940841 dv-cinder-841125-4-iscsi devstack@keystone.service[80413]: [pid: 80413|app: 0|req: 1456/2951] 10.222.0.58 () {58 vars in 1232 bytes} [Tue May 10 15:01:27 2022] DELETE /identity/v3/projects/6c0b09ddfb1b412199b168462e4a988d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 10 15:06:09.000489 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Authenticating user token {{(pid=80414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 10 15:06:09.001906 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:09.097097 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366244cc-b69d-4cd0-a2eb-e08156de621c admin admin] Validating token access rules against request {{(pid=80414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 10 15:06:09.156044 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639566a1a9de49ef841486e9e21a2116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dc84015f94210b74f03308ecf7a96', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 10 15:06:09.156697 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] REQUEST_METHOD: `GET` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 10 15:06:09.156979 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] SCRIPT_NAME: `/identity` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 10 15:06:09.157255 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 10 15:06:09.159029 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.fernet_utils [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 10 15:06:09.171501 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 10 15:06:09.173068 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e43da6-5c7e-4db5-9e58-5daa23385e05 service placement] RBAC: Authorization granted {{(pid=80414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 10 15:06:09.183348 dv-cinder-841125-4-iscsi devstack@keystone.service[80414]: [pid: 80414|app: 0|req: 1496/2952] 10.222.0.58 () {60 vars in 1365 bytes} [Tue May 10 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)